Image backup processing/mount problems

artyomtsybulkin · April 22, 2023, 11:06am

Server: 2.5.30 (tested on Fedora 37/38, Ubuntu 22.04/23.04)
Server is Linux Hyper-V VM with 128Gb ext4 OS drive and 16Tb btrfs backup drive.
Client: 2.5.23 (Fresh Windows 2022 Hyper-V VM, 64Gb single drive, backup from VM)
Backup destination:

Image backup processing problem:
Configured: backup into VHD (but backup disabled) in General and into VHD in group Daily (backup enabled) - but server saves RAW files instead.

root@vbs1:~# ls /mnt/sdb1/backups/vhsw/230422-1038_Image_C
Image_C_230422-1038.raw         Image_C_230422-1038.raw.cbitmap  Image_C_230422-1038.raw.mbr
Image_C_230422-1038.raw.bitmap  Image_C_230422-1038.raw.hash     Image_C_230422-1038.raw.sync
root@vbs1:~#

Note: No matter that File backups intervals and Inage Backups intervals disabled, backups still processing.

Fix: after I removed group backup to VHD works correctly. And VHD mount works. This content I can see when VHD image mounted.

root@vbs1:~# ls /mnt/sdb1/backups/vhsw/230422-1044_Image_C/
ls: cannot access '/mnt/sdb1/backups/vhsw/230422-1044_Image_C/contents0': Permission denied
ls: cannot access '/mnt/sdb1/backups/vhsw/230422-1044_Image_C/device0': Permission denied
Image_C_230422-1044.vhd          Image_C_230422-1044.vhd.hash  Image_C_230422-1044.vhd.sync  device0
Image_C_230422-1044.vhd.cbitmap  Image_C_230422-1044.vhd.mbr   contents0

Note: switch to VHDX - helpless, anyway when client placed inside group image backup always do RAW files, no matter what configured. Nothing in logs, only warnings that full backups doing because not found. Image backup file type matters only when client doesn’t reside in any group, VHD and VHDX backup works in this case.

Image mount issues: RAW and VHDX - image mount not working in Web UI while VHD mounts fine.

Log:

2023-04-22 10:55:32: ERROR: Image mounting failed: Guestmount...
libvirt:  error : internal error: libvirt.so is not safe to use from setuid/setgid programs
libguestfs: error: /usr/bin/supermin exited with error status 1.
To see full error messages you may need to enable debugging.
Do:
  export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again.  For further information, read:
  http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.

root@vbs1:~# urbackup_mount_helper test
MOUNT TEST OK

root@vbs1:~# libguestfs-test-tool
...
===== TEST FINISHED OK =====

There is a lot of lines in second test, but finish with OK status. Also, I found that tests done successfully for EXT file systems and nothing about NTFS.

Mount from command line:

root@vbs1:~# urbackupsrv mount-vhd --file /mnt/sdb1/backups/vhsw/230422-1053_Image_C/Image_C_230422-1053.raw --moun
tpoint /mnt/test/
Loading FUSE kernel module...
Starting VHD background process...
Waiting for background process to become available...
Mounting...
Mounted successfully.
root@vbs1:~# ls /mnt/test/
'$Recycle.Bin'             DumpStack.log.tmp  'Program Files (x86)'  'System Volume Information'   pagefile.sys
'$WinREAgent'              PerfLogs            ProgramData            Users
'Documents and Settings'  'Program Files'      Recovery               Windows
root@vbs1:~#

VHDX mount issues too:

While from bash:

vbsadmin@vbs1:~$ tail /var/log/urbackup.log
  export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again.  For further information, read:
  http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.

2023-04-22 12:21:52: ERROR: Error while downloading update file from https://hndl.urbackup.org/Client/2.5.23/update/UrBackupUpdate.exe: SSL peer certificate or SSH remote key was not OK(ec=60), Peer's Certificate issuer is not recognized.
2023-04-22 12:21:53: ERROR: Error while downloading update file from https://hndl.urbackup.org/Client/2.5.23/update/UrBackupUpdateLinux.sh: SSL peer certificate or SSH remote key was not OK(ec=60), Peer's Certificate issuer is not recognized.
2023-04-22 12:23:55: ERROR: Footer checksum wrong. Switching to header
2023-04-22 12:23:55: ERROR: Header and footer checksum wrong
vbsadmin@vbs1:~$ sudo urbackupsrv mount-vhd --file /mnt/sdb1/backups/vhsw/230422-1221_Image_C/Image_C_230422-1221.vhdx --mountpoint /home/vbsadmin/Mounts/
Loading FUSE kernel module...
Starting VHD background process...
Waiting for background process to become available...
Mounting...
Mounted successfully.
vbsadmin@vbs1:~$ ls Mounts/
'$Recycle.Bin'             DumpStack.log.tmp  'Program Files (x86)'  'System Volume Information'   pagefile.sys
'$WinREAgent'              PerfLogs            ProgramData            Users
'Documents and Settings'  'Program Files'      Recovery               Windows
vbsadmin@vbs1:~$

Messages about Footer and header related to VHDX mount from UI.

These things worked unstable on prior versions of urBackup Server, and now doesn’t.

Any idea?

artyomtsybulkin · April 22, 2023, 11:42am

Assuming that image mounts done by non-root, I uncommented this line in file (with server reboot):

root@vbs1:~# cat /etc/fuse.conf
# The file /etc/fuse.conf allows for the following parameters:
#
# user_allow_other - Using the allow_other mount option works fine as root, in
# order to have it work as user you need user_allow_other in /etc/fuse.conf as
# well. (This option allows users to use the allow_other option.) You need
# allow_other if you want users other than the owner to access a mounted fuse.
# This option must appear on a line by itself. There is no value, just the
# presence of the option.

user_allow_other


# mount_max = n - this option sets the maximum number of mounts.
# Currently (2014) it must be typed exactly as shown
# (with a single space before and after the equals sign).

#mount_max = 1000

And now RAW mounts in Web UI works until next reboot. After second reboot I got same message:

Under root from bash mount works via urbackupsrv mount-vhd

And problem with image backup file format still exists. Server do RAW instead of VHD/VHDX when client in group.

artyomtsybulkin · April 22, 2023, 11:45am

I have another one question: If I will switch to RAW image backup format, then how can I mount this images without urbackup web ui or urbackupsrv mount-vhd?

artyomtsybulkin · April 22, 2023, 5:27pm

Update.
Done same tests on Fedora 38 with urbackup-server package for Fedora 36.
In this case no issues with Group/Image backup file type: backup done to vhdx for client in group.
But same issue with mount from Web UI.

But from bash:

[root@vbs2 vhsw]# mkdir /mnt/sdb1/mnt
[root@vbs2 vhsw]# cd /mnt/sdb1/mnt
[root@vbs2 mnt]# ls
[root@vbs2 mnt]# pwd
/mnt/sdb1/mnt
[root@vbs2 mnt]# urbackupsrv mount-vhd --file /mnt/sdb1/backups/vhsw/230422-1320_Image_C/Image_C_230422-1320.vhdx -
-mountpoint /mnt/sdb1/mnt/
Loading FUSE kernel module...
Starting VHD background process...
Waiting for background process to become available...
Mounting...
Mounted successfully.
[root@vbs2 mnt]# ls -aghl
total 16K
drwxr-xr-x. 1 root  0 Apr 22 13:25 .
drwxr-xr-x. 1 root 20 Apr 22 13:25 ..
[root@vbs2 mnt]# ls
[root@vbs2 mnt]# cd ..
[root@vbs2 sdb1]# ls
backups  mnt
[root@vbs2 sdb1]# ls mnt/
'$Recycle.Bin'             DumpStack.log.tmp   ProgramData            Recovery                     Windows
'$WinREAgent'              pagefile.sys       'Program Files'        'System Volume Information'
'Documents and Settings'   PerfLogs           'Program Files (x86)'   Users
[root@vbs2 sdb1]#

artyomtsybulkin · April 22, 2023, 8:04pm

Configured image backup VHD/VHDX/RAW, using ALL/ALL_NONUSB.
I have drive C on Windows client. I made some test backups. Reboot client and server.
Then I added new drive H 32Gb. No matter what I do in configuration backup of second drive doesn’t processing. I have only drive C.

DrRSatzteil · April 23, 2023, 10:57am

Just one note regarding these mounting problems: I always had these and never got it fixed. However I can leave the page and return to the same image and I see the mounted files. So it does work but the error message is misleading…

artyomtsybulkin · April 23, 2023, 4:34pm

Yeah. Just made some test on fresh ArchLinux. Found, that paths problems depend on client side. Same issues with mounts, but from bash mounts work quickly without problems under root account.

artyomtsybulkin · May 8, 2023, 11:46am

Assume/hope issues except mounting VHDX, VHDZ, VHDXZ, RAW fixed in 2.5.31.