Server 2.5.21/Client 2.5.15

New settings handling

The major change with server 2.5.x and client 2.5.x is a complete redesign how settings are handled. The goal is to make it more explicit and consistent.

  • Each setting has a button next to it which switches between inheriting the setting from the group, using the configuration value on the configuration page and using the value from the client, and for some setting merging client and server settings together
  • File backup paths are now a normal setting just like any other. No more separation between paths to backup configured on the client and default directories to backup

This is a critical area of UrBackup and as such needs a lot of testing. Focus should be on if the 2.4.x client still more or less works with the 2.5.x server (and especially doesn’t loose its connection settings for Internet servers).

Connect via web sockets

With 2.5.x clients can connect via web sockets to the server. Setup should be easy. Instead of “Internet server name/IP” of e.g. “example.com” use “ws://example.com:55414/socket”. The real advantage comes into play if a real web server with SSL is setup in front. Either proxy the whole page, or serve the files and only proxy “/x” via FastCGI (see manual), additionally proxy “/socket” via web socket to “ws://127.0.0.1:55414/socket”. Then setup the clients to connect to “Internet server name” “wss://example.com/socket”. Basic authentication, a different name and a different port should also work, e.g. “wss://username:password@example.com/socket”.

Come back of macOS client

A test version of the macOS client is here. Please add your feedback!

Linux image backups

2.5.x can backup Linux clients (via dattobd or Linux device mapper snapshots). The use case is e.g. backing up a DigitalOcean instance. It is explicitly not to create image backups of all possible configurations e.g. LVM, mdraid, btrfs, …
It can “live restore” a root volume, i.e., overwrite the root volume of a currently running machine. So one can image backup e.g. a DigitalOcean droplet. To restore create a new instance (tested with Debian 10/buster) then run the restore script which overwrites the current instance with the backed up instance using a trick to put the currently running system files into (compressed) memory.
Dattobd and the Linux device mapper supports Changed Block Tracking (CBT), so image (and file) backups are done with CBT.

The Linux device mapper snapshots scripts are currently only tested on Debian 10 and with the root volume. It needs to hook into the boot sequence (initramfs) to backup the root volume. For other volumes it probably needs to hook into udev. Help getting it running on other distributions/with non-root volumes would be welcome!

Linux image backup howto

Backup:

  • To only backup used space install partclone (e.g. apt install partclone). This is currently supported for ext* and xfs. If partclone is not present it’ll backup unused space.
  • Install the (binary) client and select either dattobd or the Linux device mapper for snapshotting
  • If you are using dattobd: Install dattobd dattobd/INSTALL.md at master · datto/dattobd · GitHub

Restore:

  • On the server browse to the image backup to restore
  • Click on “Restore Linux image”
  • Copy & paste the command into a new instance and follow the instructions (the command has a timeout (token), so after a while it should not work anymore)

Testing (image backup + file backup with CBT)

To test if CBT works correctly with file backups please enable Debugging: Verify file backups using client side hashes (for Internet clients with client-side hashing) or Debugging: End-to-end verification of all file backups in the advanced settings.

To test image backups enable md5sums creation on the client via touch /usr/local/var/create_md5sums_imagebackup. The client will then create a file (with date+time in the name) at /usr/local/var/md5sums-* for each image backup listing all files in the image plus their md5sum. Copy that file to server, mount the image backup then verify the md5sums with cd /backup/client/image_mnt; md5sums --quiet -c /path/to/md5sums-file.txt. The only files with checksum errors should be the .datto-*/.overlay-* files in the root directory and swap files (those get automatically excluded from backup).


Thanks for any help with testing and feedback!

Upgrade process

As always: Replace the executables (via the installers) and the database of the server/client will be updated on first running it.

Downgrade process (server)

Stop the UrBackup server, restore C:\Program Files\UrBackupServer\urbackup or /var/urbackup from a backup before upgrade and then install the previous version over the beta release.


On testing

Please see Having problems with UrBackup? Please read before posting for what kind of information to add when reporting issues. Reports such as “X does not work” are useless. In general, at the very least, think about what kind of information one would need to reproduce the problem in a different environment. Best case you debug the problem yourself and then post the fix.


Changes with client 2.5.15

  • Start filesrv before accepting commands
  • Don’t update settings on upgrade if no paths are configured and setting file doesn’t exist yet
  • Fix waiting for connection to server when restoring image
  • Cleanup snapshot later if it cannot be deleted because it is in use
  • Build glibc client build with older glibc

Changes with server 2.5.21

  • Make vhdx support compile with older compilers
  • Build Debian package on older Debian so it works on more versions
  • Handle merge settings which cannot be configured on client
  • Select correct use value for old setting config

Changes with server 2.5.20 beta

  • Add vhdx support
  • Use GPT partitioning for larger volume images

Downloads

1 Like

Get this error and warnings when trying to compile this server on TrueNAS 12.0.U4. Urbackup 2.4.13 works flawlessly with downgraded cryptopp-5.6.5_4. Anything I can do to fix this. Thanks.

Hi, with the listed server/client combo, migrated from the last release, I have trouble starting increment image backups - full image and file full/incremental works fine.
Tried deleting last incremental, shifting to VHDX, fresh full, incremental after: stuck:

→ 07/14/21 20:46 INFO Basing image backup on last full image backup
urbackup log only on server only shows a whole bunch of “2021-07-14 21:04:05: ERROR: Sending broadcast failed! (ipv6)” lines. Mh!

Any thoughts?

Rgds
Stefan

Update - Same issue after switching to a new raw full backup. Incremental backup stuck at 0%. So it is apparently not a matter of the image type.

I can confirm what @Stefan_711 says with the incremental backups, same here.
Another Bug is that i cant change any Settings while when i click save a java? Popup pops in and says the Server Url is wrong. But its ok. Same when the Url Field is complete empty.

When creating a new computer to backup, I cannot download the Linux installer. I receive this error when building the installer: ERROR: Replacing data in install file failed

Linux incremental file backups don’t work under 2.5
Get error:

Errors 08/09/21 10:09 Could not find mountpoint of snapshot of client backup3 path 210809-1008
Errors 08/09/21 10:09 Backup had an early error. Deleting partial backup.

Using ZFS for storage. Linux clients work fine under 2.4

This machine is using a mirrored boot disk (/md0) Thought that might be the problem, but linux client with mirrored boot drive works fine under 2.4

Here is full backup log:

Info 08/09/21 10:08 Starting scheduled incremental file backup…
Warnings 08/09/21 10:09 Restarting shadow copy of / because it was started by this server
Info 08/09/21 10:09 Unmounting /dev/datto0 at /mnt/urbackup_snaps/6265d854969cfb9c880be26055ebf5df01a2e81331b1bc8c…
Info 08/09/21 10:09 Removing devicemapper snapshot…
Info 08/09/21 10:09 Transitioning dattobd snapshot /dev/datto0 to incremental…
Info 08/09/21 10:09 Snapshotting device /dev/md0 via dattobd…
Info 08/09/21 10:09 Trying to transition /dev/datto0 to snapshot…
Info 08/09/21 10:09 dbdctl transition-to-snapshot ‘//.datto_3d41c58e-6724-4d47-8981-11c766a08a24_26125a9de6f517a21dada10b87297bf793976c669a39fdb4’ 0
Info 08/09/21 10:09 Transitioned /dev/datto0 to snapshot.
Info 08/09/21 10:09 Mounting /dev/mapper/wsnap-26125a9de6f517a21dada10b87297bf793976c669a39fdb4…
Info 08/09/21 10:09 Using datto change information from //.datto_3d41c58e-6724-4d47-8981-11c766a08a24_6265d854969cfb9c880be26055ebf5df01a2e81331b1bc8c
Info 08/09/21 10:09 Scanning for changed hard links on volume of “/”…
Info 08/09/21 10:09 Indexing of “rootfs” done. 22766 filesystem lookups 0 db lookups and 0 db updates
Info 08/09/21 10:09 backup3: Loading file list…
Info 08/09/21 10:09 backup3: Calculating file tree differences…
Info 08/09/21 10:09 backup3: Creating snapshot…
Errors 08/09/21 10:09 Could not find mountpoint of snapshot of client backup3 path 210809-1008
Errors 08/09/21 10:09 Backup had an early error. Deleting partial backup.

It’s an issue on the server.

urbackup_snapshot_helper 1 mountpoint backup3 210809-1008

fails. Could you run that manually (as urbackup user) and maybe see why it fails?

get error:

cannot open ‘tank/urbackup/images/backup3/210809-1008’: dataset does not exist

(I have not done any image backups for that machine so the image dataset doesn’t exist)

Sorry, try:

urbackup_snapshot_helper 2 mountpoint backup3 210809-1008

get error:

cannot open ‘tank/urbackup/files/backup3/210809-1008’: dataset does not exist

This is the dataset that does exist:

tank/urbackup/files/backup3/210809-1008.startup-del

I see, thanks for the heads up!

In server gui, image backup status for linux clients says “disabled” even if image backups are enabled.