Client calculated hash differs from server calculated hash

client
1

Client:
Ubuntu 14.04
UrBackup Client Backend v2.0.33.0

Server:
Ubuntu 16.04
UrBackup Server v2.0.34.1611

My client performed a full backup initially. That finished properly, although I now see in the logs, errors such as:
Creating snapshot of “/www” failed
Snapshotting device /dev/vdd1 via dattobd…
Using /dev/datto1…
driver returned an error performing specified action. check dmesg for more info: Invalid argument
Creating snapshot of “www” failed.

Now my incremental backups are not working. I wonder if there was an issue with dattodb, which could now be resolved. However - I’m not entirely sure how to test dattodb?

An incremental backup fails and I now see the following:
Creating snapshot of “/run/resolvconf/resolv.conf” failed
Snapshotting device tmpfs via dattobd…
Using /dev/datto2…
driver returned an error performing specified action. check dmesg for more info: Invalid argument
Creating snapshot of “/run/network” failed
Snapshotting device tmpfs via dattobd…
Using /dev/datto2…
driver returned an error performing specified action. check dmesg for more info: Invalid argument
Client calculated hash of “/backup/urbackup/puzzle/160923-1720/www/.datto_a6397fb09ab7afc8109af40ee4c432ff7f487e450b640935” differs from server calculated hash. This may be caused by a bug or by random bit flips on the client or server hard disk. Failing backup. (Hash: tree-sparse, client hash: bO8lWFUvGmy3ODP2WC4EkYVjmUig04r7mTvTv4glfwjHpNVlAPFiA5nqieK3JgM0pK5TLX5GYVeB33scX9CQyw==, server hash: tEGgfk51wQ4NvKLJhKzljp810l6zRV4vlIKLv95uJKrcDlvG//4UkO1JRybKqCgAH/a7zhBuynowy8h7cwOLWg==)
Fatal error during backup. Backup not completed
FATAL: Backup failed because of disk problems (see previous messages)
Backup failed

dmesg shows:
[ 1915.969218] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
[ 1958.526375] EXT4-fs (dm-1): mounted filesystem with ordered data mode. Opts: (null)
[ 1959.190485] datto: illegal to perform setup while unmounted: -22
[ 1959.190523] datto: error during setup ioctl handler: -22
[ 1970.644520] datto: illegal to perform setup while unmounted: -22
[ 1970.644554] datto: error during setup ioctl handler: -22

Is this an issue with datto, or has something else gone wrong?

2

I have just upgraded the client to Ubuntu 16.04. As part of this, after booting into 16.04, I then completely removed datto and re-installed it, before rebooting once more.

Unfortunately the same error persists - the client calculated hash differs from that of the server.

3

I don’t think dattobd works with tmpfs. And it does not make much sense to backup tmpfs… It is probably some symlinks pointing there?

I should not fail the backup after hash failures without snapshot, though. That will be fixed.

4

I’ve just checked regarding symlinks pointing to tmpfs. That was indeed the cause. I was backing up /etc and there are symlinks there for /run and /dev. I’ve changed the backup include to be more specific, to /etc/cron.d/ and that has solved the issue for me.

Of course I’m still getting the ‘client calculated hash’ issue. There are no errors regarding snapshots?

The log today, shows:
Client calculated hash of “/backup/urbackup/puzzle/160926-0904/www/.datto_a6397fb09ab7afc8109af40ee4c432ff7f487e450b640935” differs from server calculated hash. This may be caused by a bug or by random bit flips on the client or server hard disk. Failing backup. (Hash: tree-sparse, client hash: bO8lWFUvGmy3ODP2WC4EkYVjmUig04r7mTvTv4glfwjHpNVlAPFiA5nqieK3JgM0pK5TLX5GYVeB33scX9CQyw==, server hash: tEGgfk51wQ4NvKLJhKzljp810l6zRV4vlIKLv95uJKrcDlvG//4UkO1JRybKqCgAH/a7zhBuynowy8h7cwOLWg==)

Fatal error during backup. Backup not completed
FATAL: Backup failed because of disk problems (see previous messages)
Backup failed

It looks to me like the snapshot was working properly this time? I could see the datto_snap_cow0 process was busy throughout the backup.

Do you have any suggestions for how to get the backup running in the meantime? Should I try removing and re-adding the client?

5

Could you add an */.datto_: and */.overlay_: exclude? Those files are excluded automatically but due to a bug only if it is in the root folder (will be fixed).

6

Yes! That has solved the problem. Thank you.