After some testing I can confirm that the Mac OS client really has a problem if it runs on Mac OS 10.13 High Sierra with APFS. I can not confirm if APFS is the problem or Mac OS 10.13 itself, as I only have 10.13 clients on APFS and one 10.12 client on a regular HFS harddisk.
On 10.12, the client runs perfectly, backups and restores are fine!
However, on 10.13 it fails to backup correctly - it looks like only the file metadata is transferred, BUT NOT THE FILES THEMSELVES. This symptom fools the web UI, everything looks perfect (although you might notice that making a full 40GB backup in 1 minute might be a bit fast for a server running on a raspberry pi ;-). However, if you try to restore files, you discover that alle files are empty. You’ll end up with files having the right size but filles with zero’s…
So, who is the Mac OS client ‘chef sache’? We need an update
Thanks for the heads up. You could help by uploading/sending a dtruss trace of a backup, as the issue is probably caused by some system calls not working as expected. For example HPFS did not support sparse files, but the client supports it (for Linux). If the sparse file calls do not work like in Linux there could be a problem where it detects the whole file as empty.
Another thing to do would be to integrate it with APFS snapshots.
I have done a 1 file full backup (look for the Simply Red mp3 file in the DTRUSS trace I cut out the DTRUSS info - here it is…I hope you can filter the problem. My guess is that there is some nice info and errors in it
Thanks in advance - keep up the good work!
BTW, it is an rtf file but I renamed it to a .txt.
5200/0x659d3: open("/Users/rene/Documents/UrBackupTest/Simply Red - Love Gave Me More.mp3\0", 0x1000000, 0x0) = 43 0
5200/0x63740: lstat64("/Users/rene/Documents/UrBackupTest/Simply Red - Love Gave Me More.mp3\0", 0x7000103984F0, 0x0) = 0 0
5200/0x659d3: fstat64(0x2B, 0x70000FEFCFE0, 0x0) = 0 0
5200/0x659d3: lseek(0x2B, 0x0, 0x4) = 0 0
5200/0x63740: lstat64("/Users/rene/Documents/UrBackupTest/Simply Red - Love Gave Me More.mp3\0", 0x7000103984A8, 0x0) = 0 0
5200/0x659d3: lseek(0x2B, 0x0, 0x3) = 7887081 0
5200/0x63740: stat64("/Users/rene/Documents/UrBackupTest/Simply Red - Love Gave Me More.mp3\0", 0x7000103984B8, 0x0) = 0 0
5200/0x659d3: lseek(0x2B, 0x7858E9, 0x4) = -1 Err#6\
Hi @uroni,
we have an Apple developer account in our company, not sure if this could help here somehow,
but I could try for sure.
Just can confirm now, too, that backups of files from Macs with 10.13 seem successful - but on restore,
all files which have been backed up after Upgrade to High Sierra … contain only zeros.
So this is a really high risk problem for us.
AFAIK the maintainers of Carbon Copy Cloner seem to have a lot of knowledge about APFS, e.g.
Fixing this is trivial – just remove the sparse file support from the Apple client.
I was hinting at the other option – Apple fixing the API such that it works the same as in Linux/FreeBSD, because there is no reason to make it inverse.
That’d be great for now - so we have a reliable backup again. Are you sure this is only a sparse file problem?
When I made my restore tests, there were zip files containing zeros, too - and IMHO these are no sparse files. Or can this be another problem?
So, how could I test this, can we try the next client beta with stable server, or is it needed to upgrade both server and client to beta versions?
Well, found the following things for now that seem related:
But there’s no apfs_snapshot utility like mentioned in the last link:
bash-3.2# ls -l /System/Library/Filesystems/apfs.fs/Contents/Resources/
total 6240
-rwxr-xr-x 1 root wheel 526784 Oct 25 18:37 apfs.util
-rwxr-xr-x 1 root wheel 529952 Oct 25 18:37 apfs_invert
-rwxr-xr-x 1 root wheel 1271328 Oct 25 18:37 apfs_preflight_converter
-rwxr-xr-x 1 root wheel 24704 Oct 25 18:37 apfs_stats
-rwxr-xr-x 1 root wheel 694560 Oct 25 18:37 fsck_apfs
-rwxr-xr-x 1 root wheel 1265504 Oct 25 18:37 hfs_convert
-rwxr-xr-x 1 root wheel 34672 Oct 25 18:37 mount_apfs
-rwxr-xr-x 1 root wheel 559248 Oct 25 18:37 newfs_apfs
-rwxr-xr-x 1 root wheel 520736 Oct 25 18:37 slurpAPFSMeta
Using the tmutil snapshots would be cool, but … may be this works only if another Time Machine volume is configured:
bash-3.2# /System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util
Usage: apfs.util [[-p][-k] /dev/diskXsYsZ]|[-s /dev/diskXsY]|[-R SNAPSHOT][-M dir][-S dir][-O path]
-p : probe for volume name
-k : get volume UUID
-s : set volume UUID(s) and container UUID to new random values
-R SNAPSHOT : set the volume to revert to the snapshot named by SNAPSHOT on next mount.
-M dir : flag the named directory as “maintain-dir-stats”.
-S dir : get the directory stats from dir.
-O path [optional fs name] : override the fstypename for apfs to be “hfs” (or the optional name given)
apfs.util [-D [get | set=yes | set=no] /dev/diskXsY] -D get : Get defragmentation setting for volume or container
-D set=yes : Enable defragmentation for volume or container
-D set=no : Disable defragmentation for volume or container
apfs.util -B
-B : bootstrap the root filesystem
So maybe we can put together the apfs_create_filesystem_snapshot script using these cli tools.
Cannot do the tests on a production system, tho - but I*ll try to setup a test machine with High Sierra. If I read it correctly, the apfs tools have been available since Sierra, so one could create an apfs volume using Sierra and make some tests ,
You should be able to test this by creating a /Applications/UrBackup Client.app/Contents/MacOS/etc/urbackup/snapshot.cfg file with contents pointing to snapshot creation and destruction scripts: