We have a fresh machine (tried on two separate instances), with AlmaLinux9, Dattobd. The sistem has XFS filesystems on LVM.
Fstab:
/dev/mapper/almalinux-root / xfs defaults 0 0
UUID=eb5bb418-904e-4e57-b536-931c9194db56 /boot xfs defaults 0 0
/dev/mapper/almalinux-swap none swap defaults 0 0
/dev/data/mail /var/vmail xfs defaults,inode64 0 0
Urbackup client log:
2024-03-20 12:42:54: ERROR: Creating snapshot of "/etc" failed
2024-03-20 12:42:54: ERROR: Snapshotting device /dev/mapper/almalinux-root via dattobd...
2024-03-20 12:42:54: ERROR: Using /dev/datto1...
2024-03-20 12:42:54: ERROR: CBT=type=datto&reset=1
2024-03-20 12:42:54: ERROR: Mounting /dev/mapper/wsnap-6323d02f9b2f8d3da61b5b371b3837353711408b3a9a5d13...
2024-03-20 12:42:54: ERROR: mount: /mnt/urbackup_snaps/6323d02f9b2f8d3da61b5b371b3837353711408b3a9a5d13: wrong fs type, bad option, bad superblock on /dev/mapper/wsnap-6323d02f9b2f8d3da61b5b371b3837353711408b3a9a5d13, missing codepage or helper program, or other error.
2024-03-20 12:42:54: ERROR: Mounting filesystem failed
2024-03-20 12:42:54: ERROR: Creating snapshot of "etc" failed.
2024-03-20 12:43:04: ERROR: Snapshotting device /dev/mapper/almalinux-root via dattobd...
2024-03-20 12:43:04: ERROR: Using /dev/datto2...
2024-03-20 12:43:04: ERROR: CBT=type=datto&reset=1
2024-03-20 12:43:04: ERROR: Mounting /dev/mapper/wsnap-5103cc0ae9e8d3733e166a50bbd3d3cabc649f53267d2af4...
2024-03-20 12:43:04: ERROR: mount: /mnt/urbackup_snaps/5103cc0ae9e8d3733e166a50bbd3d3cabc649f53267d2af4: wrong fs type, bad option, bad superblock on /dev/mapper/wsnap-5103cc0ae9e8d3733e166a50bbd3d3cabc649f53267d2af4, missing codepage or helper program, or other error.
2024-03-20 12:43:04: ERROR: Mounting filesystem failed
2024-03-20 12:43:14: ERROR: Snapshotting device /dev/mapper/almalinux-root via dattobd...
2024-03-20 12:43:14: ERROR: Using /dev/datto2...
2024-03-20 12:43:14: ERROR: CBT=type=datto&reset=1
2024-03-20 12:43:14: ERROR: Mounting /dev/mapper/wsnap-a4608c6e4dd22a157433a49decaeaea26017efd76fa1feb9...
2024-03-20 12:43:14: ERROR: mount: /mnt/urbackup_snaps/a4608c6e4dd22a157433a49decaeaea26017efd76fa1feb9: wrong fs type, bad option, bad superblock on /dev/mapper/wsnap-a4608c6e4dd22a157433a49decaeaea26017efd76fa1feb9, missing codepage or helper program, or other error.
2024-03-20 12:43:14: ERROR: Mounting filesystem failed
2024-03-20 12:43:23: ERROR: Creating snapshot of "/usr/lib/os-release" failed
2024-03-20 12:43:23: ERROR: Snapshotting device /dev/mapper/almalinux-root via dattobd...
2024-03-20 12:43:23: ERROR: Using /dev/datto2...
2024-03-20 12:43:23: ERROR: CBT=type=datto&reset=1
2024-03-20 12:43:23: ERROR: Mounting /dev/mapper/wsnap-15f50c18b959c19a4d21b991cfb16a85f2a7bcd24dfd7f79...
2024-03-20 12:43:23: ERROR: mount: /mnt/urbackup_snaps/15f50c18b959c19a4d21b991cfb16a85f2a7bcd24dfd7f79: wrong fs type, bad option, bad superblock on /dev/mapper/wsnap-15f50c18b959c19a4d21b991cfb16a85f2a7bcd24dfd7f79, missing codepage or helper program, or other error.
2024-03-20 12:43:23: ERROR: Mounting filesystem failed
DMESG:
[Wed Mar 20 12:42:43 2024] loop: module loaded
[Wed Mar 20 12:42:43 2024] loop0: detected capacity change from 0 to 204800
[Wed Mar 20 12:42:43 2024] XFS (dm-3): Mounting V5 Filesystem
[Wed Mar 20 12:42:43 2024] XFS (dm-3): Starting recovery (logdev: internal)
[Wed Mar 20 12:42:43 2024] XFS (dm-3): Ending recovery (logdev: internal)
[Wed Mar 20 12:42:52 2024] loop1: detected capacity change from 0 to 204800
[Wed Mar 20 12:42:52 2024] XFS (dm-4): Mounting V5 Filesystem
[Wed Mar 20 12:42:53 2024] XFS (dm-4): Corruption warning: Metadata has LSN (3:95282) ahead of current LSN (3:95265). Please unmount and run xfs_repair (>= v4.3) to resolve.
[Wed Mar 20 12:42:53 2024] XFS (dm-4): log mount/recovery failed: error -22
[Wed Mar 20 12:42:53 2024] XFS (dm-4): log mount failed
[Wed Mar 20 12:42:53 2024] dm-4: detected capacity change from 58286080 to 0
[Wed Mar 20 12:42:53 2024] datto: error unlinking file: -1
[Wed Mar 20 12:42:53 2024] loop1: detected capacity change from 0 to 204800
[Wed Mar 20 12:42:54 2024] XFS (dm-4): Mounting V5 Filesystem
[Wed Mar 20 12:42:54 2024] XFS (dm-4): Starting recovery (logdev: internal)
[Wed Mar 20 12:42:54 2024] XFS (dm-4): Ending recovery (logdev: internal)
[Wed Mar 20 12:43:03 2024] loop2: detected capacity change from 0 to 204800
[Wed Mar 20 12:43:03 2024] XFS (dm-5): Mounting V5 Filesystem
[Wed Mar 20 12:43:03 2024] XFS (dm-5): Corruption warning: Metadata has LSN (3:95539) ahead of current LSN (3:95525). Please unmount and run xfs_repair (>= v4.3) to resolve.
[Wed Mar 20 12:43:03 2024] XFS (dm-5): log mount/recovery failed: error -22
[Wed Mar 20 12:43:03 2024] XFS (dm-5): log mount failed
[Wed Mar 20 12:43:03 2024] dm-5: detected capacity change from 58286080 to 0
[Wed Mar 20 12:43:04 2024] datto: error unlinking file: -1
[Wed Mar 20 12:43:13 2024] loop2: detected capacity change from 0 to 204800
[Wed Mar 20 12:43:13 2024] XFS (dm-5): Mounting V5 Filesystem
[Wed Mar 20 12:43:13 2024] XFS (dm-5): Corruption warning: Metadata has LSN (3:95633) ahead of current LSN (3:95631). Please unmount and run xfs_repair (>= v4.3) to resolve.
[Wed Mar 20 12:43:13 2024] XFS (dm-5): log mount/recovery failed: error -22
[Wed Mar 20 12:43:13 2024] XFS (dm-5): log mount failed
[Wed Mar 20 12:43:13 2024] dm-5: detected capacity change from 58286080 to 0
[Wed Mar 20 12:43:13 2024] datto: error unlinking file: -1
[Wed Mar 20 12:43:22 2024] loop2: detected capacity change from 0 to 204800
[Wed Mar 20 12:43:22 2024] XFS (dm-5): Mounting V5 Filesystem
[Wed Mar 20 12:43:22 2024] XFS (dm-5): Corruption warning: Metadata has LSN (3:95707) ahead of current LSN (3:95693). Please unmount and run xfs_repair (>= v4.3) to resolve.
[Wed Mar 20 12:43:22 2024] XFS (dm-5): log mount/recovery failed: error -22
[Wed Mar 20 12:43:22 2024] XFS (dm-5): log mount failed
[Wed Mar 20 12:43:23 2024] dm-5: detected capacity change from 58286080 to 0
[Wed Mar 20 12:43:23 2024] datto: error unlinking file: -1
[Wed Mar 20 12:43:32 2024] loop2: detected capacity change from 0 to 204800
[Wed Mar 20 12:43:32 2024] XFS (dm-5): Mounting V5 Filesystem
[Wed Mar 20 12:43:32 2024] XFS (dm-5): Starting recovery (logdev: internal)
[Wed Mar 20 12:43:32 2024] XFS (dm-5): Ending recovery (logdev: internal)
[Wed Mar 20 12:43:34 2024] XFS (dm-3): Unmounting Filesystem
[Wed Mar 20 12:43:34 2024] dm-3: detected capacity change from 67100672 to 0
[Wed Mar 20 12:43:35 2024] XFS (dm-4): Unmounting Filesystem
[Wed Mar 20 12:43:35 2024] dm-4: detected capacity change from 2097152 to 0
[Wed Mar 20 12:43:41 2024] XFS (dm-5): Unmounting Filesystem
[Wed Mar 20 12:43:41 2024] dm-5: detected capacity change from 58286080 to 0
Somehow the next full backup finished with NO errors. So sometimes it works, sometimes it doesn’t.
First incremental:
Urbackup client log:
2024-03-20 13:02:39: ERROR: Creating snapshot of "/etc" failed
2024-03-20 13:02:39: ERROR: Snapshotting device /dev/mapper/almalinux-root via dattobd...
2024-03-20 13:02:39: ERROR: Trying to transition /dev/datto2 to snapshot...
2024-03-20 13:02:39: ERROR: dbdctl transition-to-snapshot /datto_3d41c58e-6724-4d47-8981-11c766a08a24_a2480e8e7295ce86b980f8a634bf2a01dbe6354c2aa78c52 2
2024-03-20 13:02:39: ERROR: Transitioned /dev/datto2 to snapshot.
2024-03-20 13:02:39: ERROR: CBT_FILE=/.datto_3d41c58e-6724-4d47-8981-11c766a08a24_c8a1e01631f20cffbe2fd3a53f5e48db8524a68597600f4f
2024-03-20 13:02:39: ERROR: CBT=type=datto
2024-03-20 13:02:39: ERROR: Mounting /dev/mapper/wsnap-a2480e8e7295ce86b980f8a634bf2a01dbe6354c2aa78c52...
2024-03-20 13:02:39: ERROR: mount: /mnt/urbackup_snaps/a2480e8e7295ce86b980f8a634bf2a01dbe6354c2aa78c52: wrong fs type, bad option, bad superblock on /dev/mapper/wsnap-a2480e8e7295ce86b980f8a634bf2a01dbe6354c2aa78c52, missing codepage or helper program, or other error.
2024-03-20 13:02:39: ERROR: Mounting filesystem failed
2024-03-20 13:02:39: ERROR: Creating snapshot of "etc" failed.
DMESG:
[Wed Mar 20 13:02:23 2024] loop0: detected capacity change from 0 to 204800
[Wed Mar 20 13:02:23 2024] XFS (dm-3): Mounting V5 Filesystem
[Wed Mar 20 13:02:23 2024] XFS (dm-3): Starting recovery (logdev: internal)
[Wed Mar 20 13:02:23 2024] XFS (dm-3): Ending recovery (logdev: internal)
[Wed Mar 20 13:02:37 2024] loop1: detected capacity change from 0 to 204800
[Wed Mar 20 13:02:38 2024] XFS (dm-4): Mounting V5 Filesystem
[Wed Mar 20 13:02:38 2024] XFS (dm-4): Corruption warning: Metadata has LSN (3:97389) ahead of current LSN (3:97379). Please unmount and run xfs_repair (>= v4.3) to resolve.
[Wed Mar 20 13:02:38 2024] XFS (dm-4): log mount/recovery failed: error -22
[Wed Mar 20 13:02:38 2024] XFS (dm-4): log mount failed
[Wed Mar 20 13:02:38 2024] dm-4: detected capacity change from 58286080 to 0
[Wed Mar 20 13:02:38 2024] datto: error unlinking file: -1
[Wed Mar 20 13:02:39 2024] loop1: detected capacity change from 0 to 204800
[Wed Mar 20 13:02:39 2024] XFS (dm-4): Mounting V5 Filesystem
[Wed Mar 20 13:02:39 2024] XFS (dm-4): Starting recovery (logdev: internal)
[Wed Mar 20 13:02:39 2024] XFS (dm-4): Ending recovery (logdev: internal)
[Wed Mar 20 13:02:40 2024] datto: device specified does not exist: -2
[Wed Mar 20 13:02:40 2024] datto: error during transition to snapshot ioctl handler: -2
[Wed Mar 20 13:02:48 2024] loop2: detected capacity change from 0 to 204800
[Wed Mar 20 13:02:48 2024] XFS (dm-5): Mounting V5 Filesystem
[Wed Mar 20 13:02:48 2024] XFS (dm-5): Starting recovery (logdev: internal)
[Wed Mar 20 13:02:48 2024] XFS (dm-5): Ending recovery (logdev: internal)
[Wed Mar 20 13:02:50 2024] XFS (dm-3): Unmounting Filesystem
[Wed Mar 20 13:02:50 2024] dm-3: detected capacity change from 67100672 to 0
[Wed Mar 20 13:02:50 2024] XFS (dm-4): Unmounting Filesystem
[Wed Mar 20 13:02:50 2024] dm-4: detected capacity change from 2097152 to 0
[Wed Mar 20 13:02:56 2024] XFS (dm-5): Unmounting Filesystem
[Wed Mar 20 13:02:56 2024] dm-5: detected capacity change from 58286080 to 0
Next incremental sometimes work, but mostly not.
Also it created 10 (obviously ran it 10 times) “.datto_…” files with the size of 3GB (probably 10% of 32BG disk) in /, but didn’t delete it, so disk got full very fast.
Urbackup was quite stable on CentOS 7, never used is on any other system. But this is very unstable.
I can assist with testing, but I have no idea what to do next.
Thank you for your help!