UrBackup Server 2.1.16 beta (updated)/Client 2.1.12 beta

Major changes with server 2.1.x beta

  • Groups for client settings
  • Ability to reconnect during indexing if connection is broken
  • Define which volumes need to be snapshotted together (snapshot groups)
  • Image backup mounting, browsing and archival on Windows, Linux and FreeBSD (does not work in FreeBSD jails and with VHD/VHDZ on FreeBSD)

Major changes with client 2.1.x beta

  • Ability to reconnect during indexing if connection is broken
  • Improved image backup performance
  • Windows Backup API support (tested backup and restore with Microsoft SQL, tested backup with Microsoft Exchange and Hyper V)
  • File backups and restores use the change block data from the change block tracking driver now (you can install the beta client over a CBT client and this will work)
  • Define which volumes need to be snapshotted together (snapshot groups)

Changes with server 2.1.15 beta

  • Make failure to load backupcom.xml from client a non-fatal error
  • Align sparse extents to sparse extent block size during patching
  • Improve restore meta data error logging
  • Open path read only for syncfs (Linux)
  • Work around missing LOOP_CTL_GET_FREE ioctl definition
  • Resize cow-file bitmap if necessary. Fixed write beyond memory section if volume size is increased after first full image backup
  • Add no-cache header to ZIP and file downloads
  • Flush compression buffer after some time in ZIP file download to keep connection alive
  • Guestmount tuning (image backup mounting on Linux)
  • Correctly set loop device to use direct io (performance of image backup mounting on Linux)

Changes with server 2.1.16 beta

  • Log about start-up recovery being finished
  • Backup server_ident_ecdsa409k1.priv/server_ident_ecdsa409k1.pub in nightly backup
  • Return success if wanting to trim beyond original file size in cow-raw image backup
  • Put md5sum into css/js file names and cache based on them
  • Fallback to non-dio losetup if O_DIRECT opening does not work
  • Increase robustness of OFB block load in block-hash differences file transfer
  • Drop extra groups on Linux on startup

Changes with client 2.1.12 beta

  • Do not lower account/groupnames on Linux
  • Delete files/folders in sub-directories during file restore
  • Do not follow symlinks when deleting files during file restore
  • Continue file backup after non-selected Windows backup components writer failures
  • Do not truncate symlinks in restore
  • Create correct symlink type during file restore on Windows
  • Fix async index delayed finishing
  • Continue using CBT data if there is not enough bitmap data

Upgrade process

As always: Replace the executables (via the installers) and the database of the server/client will be updated on first running it.

Place the files from the update directory into C:\Program Files\UrBackupServer\urbackup or /var/urbackup to auto-update clients. Disable Download client from update server in the server settings to prevent the server from downloading the current version.

On Linux e.g. with this update script: https://github.com/ptempier/get_urbackupclient/blob/master/updateclient.sh

Downgrade process (server)

Stop the UrBackup server, restore C:\Program Files\UrBackupServer\urbackup or /var/urbackup from a backup before upgrade and then install the previous version over the beta release.

Downloads

Testing 2.1.15/2.1.12 restore, it seems that this version deletes newly created files after the backup. But log shows:

Info
01/18/17 19:30
Starting restore of path “Users”…
Info
01/18/17 19:30
Starting restore…
Info
01/18/17 19:30
Loading file list…
Info
01/18/17 19:30
Calculating download size…
Info
01/18/17 19:30
Downloading necessary file data…
Warnings
01/18/17 19:30
Error deleting file “C:\Users\MY\AppData\Local\Microsoft\Windows\History\History.IE5\MSHist012017012920170130\index.dat”. The process cannot access the file because it is being used by another process. (code: 32)
Warnings
01/18/17 19:30
Error deleting directory “C:\Users\MY\AppData\Local\Microsoft\Windows\History\History.IE5”. The directory is not empty. (code: 145)
Info
01/18/17 19:30
Applying file metadata…
Info
01/18/17 19:30
Restore finished successfully.

Still got a volume backup error with resized volume after upgrading to server 2.1.15 / client 2.1.12:

Regards,

Browsing images contents doesn’t work with ZFS backend :

From GUI :

From CLI :

root@backup-6:~# urbackup_mount_helper mount SVIMPORT 170130-1729_Image_C Image_C_170130-1729.raw
Error opening backing file /media/backups/SVIMPORT/170130-1729_Image_C/Image_C_170130-1729.raw

File exists :

root@backup-6:~# ll /media/backups/SVIMPORT/170130-1729_Image_C/Image_C_170130-1729.raw
-rwxr-x--- 1 root root 64055934464 janv. 30 19:00 /media/backups/SVIMPORT/170130-1729_Image_C/Image_C_170130-1729.raw
root@backup-6:~#

Regards,

EDIT : Backups are stored in an “images” dataset

root@backup-6:~# cat /etc/urbackup/dataset
backups/images
root@backup-6:~#

I think it is the missing O_DIRECT https://github.com/zfsonlinux/zfs/issues/224 . I will add a work-around for ZFS-On-Linux.

Thanks for the report! Will be fixed.

I have 2 clients running client version 2.1.12 on Windows Server 2016 (server version 2.1.15) that will not automatically backup nightly. I can manually run the full or incremental image and it works fine but on the status page it always say ‘No recent backup’.

Any suggestions on where to look to resolve this?

Here is the log from one of my clients.

Info

01/30/17 22:03
Starting scheduled full file backup…
Errors

01/30/17 22:13
Connecting to ClientService of “ICS-EXCH” failed - CONNECT error during filelist construction (2)
Errors

01/30/17 22:13
Backup had an early error. Deleting partial backup.

Updated.

uroni,

Could you please also update the source for urbackup_frontend_wx on github? I tried to compile the github source but the compiled client (for windows) stopped the service whenever I initiate a backup. I tried the precompiled 2.1.12 you posted in this thread it works well. Therefore I think there may be something changed in the frontend?
Thanks!

For information, version 2.1.16 fix the ZFS mount image bug and also the trimming crash after client image resize :sunglasses:

Thanks a lot !

It is pretty much up to date (except for incrementing the version in version.txt). This does not start/stop the service in any case.

Could you do the restore test with the failed metadata again?

uroni,

Thanks for the information.

I forgot to report that I have tested the restore with failed metadata and it seems working well this version. Thanks!
But a new issue shown up after file restore:
Warnings
01/10/17 21:00
Error deleting file “C:\Users\MY\Searches\Everywhere.search-ms_urbackup_restore_0”. Access is denied. (code: 5)
Warnings
01/10/17 21:00
Error deleting file “C:\Users\MY\Searches\Indexed Locations.search-ms_urbackup_restore_0”. Access is denied. (code: 5)
It is a minor issue and shown as “warning”. File restore was complete.

Backup works fine but restoring volume or mounting image through GUI doesn’t.

  1. Restoring volume (with urbackup_restore-2.0.3-2-amd64.iso) to an empty disk finished successfully but result in a RAW partition under Windows :

  1. “Mount image” to browse from GUI failed :

RAW file before resizing client disk (100 Gb)

root@backup-2:~# fdisk -l /media/c4d5fd17-2c8b-493b-8f6b-af8514c2e941/backups/SV407/161225-0043_Image_T/Image_T_161225-0043.raw

Disque /media/c4d5fd17-2c8b-493b-8f6b-af8514c2e941/backups/SV407/161225-0043_Image_T/Image_T_161225-0043.raw : 100 GiB, 107371560448 octets, 209710079 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets
Type d'étiquette de disque : dos
Identifiant de disque : 0x798a3301

Device                                                                                                 Boot Start       End   Sectors  Size Id Type
/media/c4d5fd17-2c8b-493b-8f6b-af8514c2e941/backups/SV407/161225-0043_Image_T/Image_T_161225-0043.raw1 *     1024 209710078 209709055  100G  7 HPFS/NTFS/exFAT

root@backup-2:~#

RAW file after resize client disk (120Gb) :slight_smile:

root@backup-2:~# fdisk -l /media/c4d5fd17-2c8b-493b-8f6b-af8514c2e941/backups/SV407/170202-2225_Image_T/Image_T_170202-2225.raw

Disque /media/c4d5fd17-2c8b-493b-8f6b-af8514c2e941/backups/SV407/170202-2225_Image_T/Image_T_170202-2225.raw : 120 GiB, 128846393344 octets, 251653112 secteurs
Unités : secteur de 1 × 512 = 512 octets
Taille de secteur (logique / physique) : 512 octets / 512 octets
taille d'E/S (minimale / optimale) : 512 octets / 512 octets
Type d'étiquette de disque : dos
Identifiant de disque : 0x271577d4

Device                                                                                                 Boot Start       End   Sectors  Size Id Type
/media/c4d5fd17-2c8b-493b-8f6b-af8514c2e941/backups/SV407/170202-2225_Image_T/Image_T_170202-2225.raw1 *     1024 251653118 251652095  120G  7 HPFS/NTFS/exFAT

root@backup-2:~#

Regards,

What does image mounting from command line say?

root@backup-2:~# urbackup_mount_helper mount SV407 170202-2225_Image_T Image_T_170202-2225.raw -u
Guestmount...
libguestfs: error: mount_options: /dev/sda on / (options: 'ro'): Failed to read last sector (251652094): Invalid argument
HINTS: Either the volume is a RAID/LDM but it wasn't setup yet,
   or it was not setup correctly (e.g. by not using mdadm --build ...),
   or a wrong device is tried to be mounted,
   or the partition table is corrupt (partition is smaller than NTFS),
   or the NTFS boot sector is corrupt (NTFS size is not valid).
Failed to mount '/dev/sda': Invalid argument
The device '/dev/sda' doesn't seem to have a valid NTFS.
Maybe the wrong device is used? Or the whole disk instead of a
partition (e.g. /dev/sda, not /dev/sda1)? Or the other way around?
guestmount: '/dev/sda' could not be mounted.
guestmount: Did you mean to mount one of these filesystems?
guestmount:     /dev/sda (ntfs)
root@backup-2:~#

Can you do truncate -s 130G Image_T_170202-2225.raw and then retry?

It fixed the problem of the GUI browsing.

...
/dev/fuse          120G    115G  5,8G  96% /media/c4d5fd17-2c8b-493b-8f6b-af8514c2e941/backups/SV407/170202-2225_Image_T
...

I’m going to try a volume restore on a 130 Gb empty virtual drive …

Restore from iso failed with this message :

You need to restore it to a bigger disk.