After some testing I can confirm that the Mac OS client really has a problem if it runs on Mac OS 10.13 High Sierra with APFS. I can not confirm if APFS is the problem or Mac OS 10.13 itself, as I only have 10.13 clients on APFS and one 10.12 client on a regular HFS harddisk.
On 10.12, the client runs perfectly, backups and restores are fine!
However, on 10.13 it fails to backup correctly - it looks like only the file metadata is transferred, BUT NOT THE FILES THEMSELVES. This symptom fools the web UI, everything looks perfect (although you might notice that making a full 40GB backup in 1 minute might be a bit fast for a server running on a raspberry pi ;-). However, if you try to restore files, you discover that alle files are empty. You’ll end up with files having the right size but filles with zero’s…
So, who is the Mac OS client ‘chef sache’? We need an update
Thanks for the heads up. You could help by uploading/sending a dtruss trace of a backup, as the issue is probably caused by some system calls not working as expected. For example HPFS did not support sparse files, but the client supports it (for Linux). If the sparse file calls do not work like in Linux there could be a problem where it detects the whole file as empty.
Another thing to do would be to integrate it with APFS snapshots.
I have done a 1 file full backup (look for the Simply Red mp3 file in the DTRUSS trace I cut out the DTRUSS info - here it is…I hope you can filter the problem. My guess is that there is some nice info and errors in it
Thanks in advance - keep up the good work!
BTW, it is an rtf file but I renamed it to a .txt.
5200/0x659d3: open(“/Users/rene/Documents/UrBackupTest/Simply Red - Love Gave Me More.mp3\0”, 0x1000000, 0x0) = 43 0
5200/0x63740: lstat64(“/Users/rene/Documents/UrBackupTest/Simply Red - Love Gave Me More.mp3\0”, 0x7000103984F0, 0x0) = 0 0
5200/0x659d3: fstat64(0x2B, 0x70000FEFCFE0, 0x0) = 0 0
5200/0x659d3: lseek(0x2B, 0x0, 0x4) = 0 0
5200/0x63740: lstat64(“/Users/rene/Documents/UrBackupTest/Simply Red - Love Gave Me More.mp3\0”, 0x7000103984A8, 0x0) = 0 0
5200/0x659d3: lseek(0x2B, 0x0, 0x3) = 7887081 0
5200/0x63740: stat64(“/Users/rene/Documents/UrBackupTest/Simply Red - Love Gave Me More.mp3\0”, 0x7000103984B8, 0x0) = 0 0
5200/0x659d3: lseek(0x2B, 0x7858E9, 0x4) = -1 Err#6\
Hi @uroni,
we have an Apple developer account in our company, not sure if this could help here somehow,
but I could try for sure.
Just can confirm now, too, that backups of files from Macs with 10.13 seem successful - but on restore,
all files which have been backed up after Upgrade to High Sierra … contain only zeros.
So this is a really high risk problem for us.
AFAIK the maintainers of Carbon Copy Cloner seem to have a lot of knowledge about APFS, e.g.
Fixing this is trivial – just remove the sparse file support from the Apple client.
I was hinting at the other option – Apple fixing the API such that it works the same as in Linux/FreeBSD, because there is no reason to make it inverse.
That’d be great for now - so we have a reliable backup again. Are you sure this is only a sparse file problem?
When I made my restore tests, there were zip files containing zeros, too - and IMHO these are no sparse files. Or can this be another problem?
So, how could I test this, can we try the next client beta with stable server, or is it needed to upgrade both server and client to beta versions?
bash-3.2# /System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util
Usage: apfs.util [[-p][-k] /dev/diskXsYsZ]|[-s /dev/diskXsY]|[-R SNAPSHOT][-M dir][-S dir][-O path]
-p : probe for volume name
-k : get volume UUID
-s : set volume UUID(s) and container UUID to new random values
-R SNAPSHOT : set the volume to revert to the snapshot named by SNAPSHOT on next mount.
-M dir : flag the named directory as “maintain-dir-stats”.
-S dir : get the directory stats from dir.
-O path [optional fs name] : override the fstypename for apfs to be “hfs” (or the optional name given)
apfs.util [-D [get | set=yes | set=no] /dev/diskXsY] -D get : Get defragmentation setting for volume or container
-D set=yes : Enable defragmentation for volume or container
-D set=no : Disable defragmentation for volume or container
apfs.util -B
-B : bootstrap the root filesystem
So maybe we can put together the apfs_create_filesystem_snapshot script using these cli tools.
Cannot do the tests on a production system, tho - but I*ll try to setup a test machine with High Sierra. If I read it correctly, the apfs tools have been available since Sierra, so one could create an apfs volume using Sierra and make some tests ,
You should be able to test this by creating a /Applications/UrBackup Client.app/Contents/MacOS/etc/urbackup/snapshot.cfg file with contents pointing to snapshot creation and destruction scripts: