Server 2.3.1 beta/Client 2.3.0 beta

beta-release
server-beta
client-beta

#1

Major changes with server 2.3.x beta

  • Use reflinks on file systems that do not have snapshot/subvolume support (only XFS for now)
  • Run backup storage checks asynchronously from status screen
  • Allow client renames in some cases
  • Preparations for special Hyper-V client with disk images instead of volume images
  • Allow client to restore its own image backups (image restore with normal client)

Changes with server 2.3.1 beta

  • Functionality to log file changes in incremental image backups
  • Seek to correct position in source file before sending when resuming image restore
  • Return error correctly when server runs out of space during file backup in a corner case
  • Only use CURLOPT_EXPECT_100_TIMEOUT_MS if curl version is high enough
  • Fix Crypto++ 6.0 byte compile issue
  • Prevent cleanup from deleting incremental image chains that cause number of incremental images to fall below min_incr_image (in testing)

Changes with server 2.3.0 beta

  • Improve space accounting if reflinking is replaced by copying
  • Alert/status fixes
  • Set json content type when returning json in web interface
  • Stop parallel hash download thread in all cases
  • Parallel file hashing fixes
  • Add Pulseway alert script (mainly as an example)
  • Improve out of space error handling
  • Log login attempts where username does not exist as failed login attempts
  • Fix settings threading issue
  • Log failed client authentications into auth log (Linux)

Changes with client 2.3.0 beta

  • Retry llistxattr with a multiple of the required memory if it fails
  • Cooperate CBT with multiple running clients
  • Don’t show restore components option if components never have been backed up
  • Fix change indicator of open files such that the symbolic link detection works for them
  • Read USNs of symbolic links and directories on Windows instead of last modified time to detect directory metadata changes
  • Fix file permission translation on Linux/MacOS
  • Fix Crypto++ 6.0 byte compile issue
  • Handle reparse tag IO_REPARSE_TAG_APPEXECLINK on Windows
  • Backup VeraCrypt boot loader
  • Improve log message if indexing is interrupted

Upgrade process

As always: Replace the executables (via the installers) and the database of the server/client will be updated on first running it.

Place the files from the update directory into C:\Program Files\UrBackupServer\urbackup or /var/urbackup to auto-update clients. Disable Download client from update server in the server settings to prevent the server from downloading the current version.

On Linux e.g. with this update script: https://github.com/ptempier/get_urbackupclient/blob/master/updateclient.sh

Downgrade process (server)

Stop the UrBackup server, restore C:\Program Files\UrBackupServer\urbackup or /var/urbackup from a backup before upgrade and then install the previous version over the beta release.

Downloads


Issue when saving settings
#2

#3

uroni–
Did you find a problem with Restore 2.2.0 beta so that you didn’t include it in the Server 2.3.1 beta posting?


#4

Where do I get more information on this alert script? I use Pulseway RMM and would like to enable the alerts.


#5

Should be kind of self-explanatory. Currently it doesn’t have the translations for the labels yet, though.
You can cross check information here: https://www.pulseway.com/restapi/#publish
And you can also see the alert script content, of course.


#6

Hi there,
I made some more tests with the server 2.3.1 beta/client 2.3.0 beta combo on Xubuntu 18.04 Bionic with a non-privileged standard user account.
While “access/restore backups” via the urbackupclientgui seems to work on the first view but it does not not work in every case on the second view.

Case 1:
Incremental file backup of the directory “Downloads”:
Screenshot_2018-09-30_16-57-54
shows this correct view in the web browser:


but it shows this when I access it via “access/restore backups” in urbackupclientgui:
Screenshot_2018-09-30_17-04-45
Please note that the .jpeg, .tar.gz and .deb files are missing/not listed.

Case 2:
I added executable permissions (chmod +x) to the .jpeg file as an example (see below) and started a further incremental file backup.
Screenshot_2018-09-30_17-08-13
shows this correct view in the web browser:


and it shows this when I access it via “access/restore backups” in urbackupclientgui:
Screenshot_2018-09-30_17-15-03
Please note, that the .jpeg is now listed and I could restore it, too. The .tar.gz and the .deb files are still missing, both have no executable permissions
So it seems that there is something strange with the permissions or some other issues that items which do not have the executable permissions set like ordinary images etc. should be listed and restorable via “access/restore backups” in the urbackupclientgui.
@uroni: could you reproduce this observations? If so, would it be possible to fix this?
Thanks in advance,
wechsler


#7

Did you run a full file backup after upgrading the client? (Otherwise it won’t update the permissions)

Also, can you have a look at the server log file for errors while browsing the directory with missing files?


#8

Hi uroni,
thanks for the quick feedback.

Yes, but to be sure I made on a virtual machine a brand new vanilla Xubuntu 18.04 installation and installed there directly urbackup server 2.3.1 beta and urbackup client 2.3.0 beta from sources. Unfortunately, the issues observed are the same as described in the posting above. For a non-privileged standard user only directories and files with executable permissions set are listed and restorable in the “accesss/restore backups” view of the urbackupclientgui.

Yes I have the server debug log as well as the client debug log for further analysis.
I hope there is a chance to track down this permissions issue.
Thanks in advance.
wechsler


Server downgrade
#9

Ok, thanks! Found the cause of the bug.

But the user not having +x makes the directories disappear from the web interface. I am a bit confused by what exactly +x does on directories on Linux (one can’t cd into directories or stat files in them if one doesn’t have +x), and am not certain that is the correct way to handle it…


#10

Good news @uroni :grinning: thanks for your effort!
Just let me know when I can test it with an updated beta version.


#11

Hello,

I update the server (jail in freenas with zfs, deduped and compressed) from 2.2.0 to 2.3.1.
All windows 10 clients are running well.

A windows server client (running as hyper-v guest) with CBT and only running with an image backup gives now an error:

Error opening VHD file “/mnt/ub_Storage/mxx/181007-1002_Image_C/Image_C_181007-1002.vhd”
Backup failed

The same with the other volume (e:) on this server.
But Image_ESP and Image_SYSVOL were backuped.

It was running before the update without errors

Someone have an idea?
Thank you.


#12

Could you please post more details? See here for e.g. how to get the server log file: Having problems with UrBackup? Please read before posting


#13

Hi Uroni,
enclosed the log when I start Urbackup.
Should I recompile it as mentioned in the log?

2018-10-08 15:04:19: Starting HTTP-Server on port 55414
2018-10-08 15:04:19: HTTP: Server started up successfully!
2018-10-08 15:04:19: Backup destination cannot handle subvolumes and snapshots. Snapshots disabled.
2018-10-08 15:04:19: Image mounting disabled: TEST FAILED: Please compile with mountvhd (./configure --with-mountvhd)
2018-10-08 15:04:19: Reflink ioctl failed. errno=14
2018-10-08 15:04:19: Broadcasting on interface IP 192.168.169.252
2018-10-08 15:04:19: UrBackup Server start up complete.
2018-10-08 15:04:19: Server started up successfully!
2018-10-08 15:04:19: Looking for old Sessions… 0 sessions
2018-10-08 15:04:20: Downloading dataplan database…


#14

You should post a log including that, not on how the server starts…


#15

Thank you uroni,
sorry, I thought the start errors are the problem:

2018-10-08 19:20:55: Starting scheduled incremental image backup of volume “C:”…
2018-10-08 19:20:55: Starting scheduled incremental image backup of volume “E:”…
2018-10-08 19:20:55: Starting scheduled full image backup of volume “SYSVOL”…
2018-10-08 19:20:55: Basing image backup on last incremental or full image backup
2018-10-08 19:20:56: Transferred 11.0634 MB - Average speed: 70.7904 MBit/s
2018-10-08 19:20:56: Updating statistics…
2018-10-08 19:20:56: Updating image stats…
2018-10-08 19:20:56: Updating file statistics…
2018-10-08 19:20:56: Done updating statistics.
2018-10-08 19:20:57: Starting scheduled full image backup of volume “ESP”…
2018-10-08 19:21:01: VHD-Parent: “/mnt/ub_storage/xxx/181002-1800_Image_E/Image_E_181002-1800.vhd”
2018-10-08 19:21:01: VHD-Parent: “/mnt/ub_storage/xxx/181001-1800_Image_E/Image_E_181001-1800.vhd”
2018-10-08 19:21:01: VHD-Parent: “/mnt/ub_storage/xxx/180929-1804_Image_E/Image_E_180929-1804.vhd”
2018-10-08 19:21:01: Corrected VHD-Parent to: “/mnt/ub_storage/xxx/181001-1800_Image_E/Image_E_180929-1804.vhd”
2018-10-08 19:21:01: ERROR: Error opening VHD file
2018-10-08 19:21:01: ERROR: Error opening Parentvhdfile “/mnt/ub_storage/xxx/181001-1800_Image_E/Image_E_180929-1804.vhd”
2018-10-08 19:21:01: ERROR: Error opening Parentvhdfile “/mnt/ub_storage/xxx/181001-1800_Image_E/Image_E_181001-1800.vhd”
2018-10-08 19:21:01: ERROR: Error opening Parentvhdfile “/mnt/ub_storage/xxx/181002-1800_Image_E/Image_E_181002-1800.vhd”
2018-10-08 19:21:01: ERROR: Error opening parent VHD
2018-10-08 19:21:01: ERROR: Error opening VHD file “/mnt/ub_storage/xxx/181008-1920_Image_E/Image_E_181008-1920.vhd”
2018-10-08 19:21:01: Transferred 64.0054 MB - Average speed: 93.6699 MBit/s
2018-10-08 19:21:01: Time taken for backing up client xxx: 5s
2018-10-08 19:21:01: ERROR: Backup failed
2018-10-08 19:21:01: Updating statistics…
2018-10-08 19:21:01: Updating image stats…
2018-10-08 19:21:01: Updating file statistics…
2018-10-08 19:21:01: Done updating statistics.
2018-10-08 19:21:02: WARNING: Exponential backoff: Waiting at least 2h 40m before next image backup
2018-10-08 19:21:04: Transferred 100.208 MB - Average speed: 124.203 MBit/s
2018-10-08 19:21:04: Updating statistics…
2018-10-08 19:21:04: Updating image stats…
2018-10-08 19:21:04: Updating file statistics…


#16

To my last reply.
I changed the servername which has to be backuped into “xxx”, but it was a name in uppercase character like “HGHJJK”.
Is that the reason?


#17

It works with 2.3.2.
Thank you


#18

Hi Uroni,
in 2.3.2 only the full image backup is working. If I make after that an incremental one, I got the same error as above:
2018-10-08 19:21:01: Corrected VHD-Parent to: “/mnt/ub_storage/xxx/181001-1800_Image_E/Image_E_180929-1804.vhd”
2018-10-08 19:21:01: ERROR: Error opening VHD file


#19

Thanks @uroni,
I just repeated a testing round with the new client 2.3.1 beta and everything works now as intended when I access and restore my backups via “access/restore backups” in urbackupclientgui on Xubuntu 18.04 Bionic as a non-privileged standard user. :grinning: