UrBackup Server/Client 2.0.4 beta (with Mac OS X client); Restore 2.0.1

Most of the changes are described in the blog:

The Linux command line interfaces have been redesigned since then as well.

This version now includes a Mac OS X client and a portable Linux client (for x86/x86_64/ARMv6/ARM64). I did not manage to get different OS X versions to run in Virtual Box. If anyone has pointers on how to do that in a comfortable and legal manner they would be appreciated.
The portable Linux client doesnā€™t have C+Ā±exception support (yet). In my tests everything was functional but this is something that must be resolved before releasing a non-beta version.

Changes since last version

  • Fixed restore
  • Improved progress reporting for image restore
  • Fixed image restore with raw images (btrfs)
  • Cope with /etc/sysconfig not being present during Linux install (untested)

Todo

  • Linux and Mac OS X client update is untested as of yet
  • UEFI/GPT testing
  • Update documentation

Compatibility with prior versions

  • 2.x server with 1.4.x client full compatibility (please report issues)
  • 2.x client with 1.4.x server works only in local network mode (not via internet mode)
  • Older client/server combinations may work but were not tested
  • 1.x restore does not work with 2.x servers (improved login method)

Upgrade process

As always: Replace the executables and the database of the server/client will be updated on first running it. As always downgrading the database version after upgrading it is not possible, so you should backup the old database files especially since this is a beta.

Because of the improved file deduplication and statistics calculation the largest server table has to be completely rebuild. This may take a few hours depending on how many file entries you have. It will show the progress on the web interface but is not usable during the upgrade process.

Linux notes:

  • The wrapper scripts start_urbackup_server and start_urbackup_client have been removed. Please use the executable directly
  • The executable has been renamed to urbackupsrv (from urbackup_srv), the client to urbackupclientbackend (from urbackup_client)
  • There is a new command line interface for the client urbackupclientctl
  • All the plugins are now statically linked into one executable. This simplifies the compilation, debugging and packaging on Linux

Run the UrBackup server on Linux with e.g. urbackupsrv run --loglevel debug

Downloads

1 Like

Hi Uroni,

Iā€™ve just upgraded will test and report any issues/bugs.

Hi Uroni,

Ok after upgrading server to 2.0.4beta Iā€™ve noticed client downloads from the web interface are no longer working giving the same signature error from earlier beta before you fixed it.

I can confirm it works on Linux and Windows on ā€œsimpleā€ paths. On ā€œDocuments and Settingsā€, I still canā€™t resume (the pop up doesnā€™t even shows and the server log says it hasnā€™t any hash for that file/path).

Just tested, worked perfectly

Tested,the configuration file is now present also on Arch Linux.

Thank you.

this problem seems to have sorted itself out so might not have been an issue client download is now working

Hi, it seems a little bug introduced itself in ā€˜urbackup_snapshot_helperā€™ since beta 2.0.>0 I straced it back to an erroneous ā€˜-cā€™ parameter being passed to ā€˜btrfs subvolume deleteā€™ :

 [pid  8227] execve("/sbin/btrfs", ["/sbin/btrfs", "subvolume", "delete", "-c", "/media/BACKUP/urbackup/testA54hj"...], [/* 0 vars */]) = 0

This makes the snapshot helper fail every btrfs test.

root@icts-q-dcvm-4:/usr/bin# ./urbackup_snapshot_helper test
Create subvolume '/media/BACKUP/urbackup/testA54hj5luZtlorr494/A'
Create a snapshot of '/media/BACKUP/urbackup/testA54hj5luZtlorr494/A' in '/media/BACKUP/urbackup/testA54hj5luZtlorr494/B'
ERROR: error accessing '-c'
Delete subvolume '/media/BACKUP/urbackup/testA54hj5luZtlorr494/A'
TEST FAILED: Removing subvolume A failed
ERROR: error accessing '-c'
Delete subvolume '/media/BACKUP/urbackup/testA54hj5luZtlorr494/B'
TEST FAILED: Removing subvolume B failed

ā€˜urbackup_snapshot_helperā€™ from beta 2.0.0.0 works fine:

root@icts-q-dcvm-4:/tmp/unpack/usr/bin# ./urbackup_snapshot_helper test
Create subvolume '/media/BACKUP/urbackup/testA54hj5luZtlorr494/A'
Create a snapshot of '/media/BACKUP/urbackup/testA54hj5luZtlorr494/A' in '/media/BACKUP/urbackup/testA54hj5luZtlorr494/B'
Delete subvolume '/media/BACKUP/urbackup/testA54hj5luZtlorr494/A'
Delete subvolume '/media/BACKUP/urbackup/testA54hj5luZtlorr494/B'
TEST OK

Update: this happens on Ubuntu 14.04, which apparently uses an older ā€˜btrfsā€™ where the ā€˜-cā€™ option to subvolume delete does not yet exist.

Hi Uroni,

Testing on Freenas with 2.0.4 clients and server.

Seems to be working mostly OK, so thank you very much for your hard work
I did have issues initially where files would only very slowly be removed from the file queue, as urbackup was searching for the file hashes of teh previous full file backup and couldnā€™t read them.

I got around this by deleting the clients and readding them since they were test machines.

Other than that, I am getting ā€œerror setting filetimeā€ warnings at the ends of Backups. Does this have anything to do with atime not being set ?

Server:

Installed on Freenas and seems to work properly

Linux Client
Testing on
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=15.10
DISTRIB_CODENAME=wily
DISTRIB_DESCRIPTION=ā€œUbuntu 15.10ā€

When installing the client gives this error:

root@erick-Latitude-E6320:/home/erick/Downloads# ./beta.sh
Verifying archive integrityā€¦ All good.
Uncompressing UrBackup Client Installer for Linux 100%
Installation of UrBackup Client 2.0.4 betaā€¦ Proceed ? [Y/n]
y
Uncompressing install dataā€¦
Detected Debian (derivative) system
Detected systemd
Detected architecture x86_64-linux-eng
Installing systemd unitā€¦
./install_client_linux.sh: 1: ./install_client_linux.sh: pkg-config: not found
root@erick-Latitude-E6320:/home/erick/Downloads#

Fixed by installing pkg-config: apt-get install pkg-config

When Selecting option 1 for snapshots it says:

W: Failed to fetch https://cpkg.datto.com/repositories/dists/wily/main/binary-amd64/Packages HttpError404

W: Failed to fetch https://cpkg.datto.com/repositories/dists/wily/main/binary-i386/Packages HttpError404

E: Some index files failed to download. They have been ignored, or old ones used instead.

When selecting option 4 for no snapshots says:

Detected Debian (derivative) system
Detected systemd
Stopping currently running client serviceā€¦
Detected architecture x86_64-linux-eng
Installing systemd unitā€¦
Starting UrBackup Client serviceā€¦
Successfully started client service. Installation complete.
+Detected Ubuntu. Dattobd supported
-Detected no btrfs filesystem
-LVM not installed
Please select the snapshot mechanism to be used for backups:

  1. dattobd volume snapshot kernel module from https://github.com/datto/dattobd
  2. Use no snapshot mechanism
    4
    Configured no snapshot mechanism

Worked on second try

Verifying archive integrityā€¦ All good.
Uncompressing UrBackup Client Installer for Linux 100%
Installation of UrBackup Client 2.0.4 betaā€¦ Proceed ? [Y/n]
y
Uncompressing install dataā€¦
Detected Debian (derivative) system
Detected systemd
Stopping currently running client serviceā€¦
Detected architecture x86_64-linux-eng
Installing systemd unitā€¦
Starting UrBackup Client serviceā€¦
Successfully started client service. Installation complete.

Full file backups work

I LOVE THE SPEED METER!!!

This is something that needs improvement. Thanks for the hint!

It seems it needs a fallback for older btrfs-tools. Thanks for the hint!

Your server probably downloaded it while I was uploading or your download got interrupted.

Ok, will have a look at this.

No, Iā€™m getting this sporatically as well but havenā€™t looked at it yet. This is only a ā€œbeautyā€ thing. It sets the file times of the files in the backup storage to the file times of the files being backed up. Those file times are then not used again. For restore and the web interface the internal metadata is directly read and used.

Hello,
with 2.0.4, Iā€™m noticing one (blocking) problem: the backup starts indexing, then hangs. The client log shows a lot of:
[ā€¦]

2016-02-13 04:13:47: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:13:47: Active query(0): END;
2016-02-13 04:13:57: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:13:57: Active query(0): END;
2016-02-13 04:14:07: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:14:07: Active query(0): END;
2016-02-13 04:14:17: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:14:17: Active query(0): END;
2016-02-13 04:14:27: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:14:27: Active query(0): END;
2016-02-13 04:14:37: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:14:37: Active query(0): END;
2016-02-13 04:14:47: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:14:47: Active query(0): END;
2016-02-13 04:14:57: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:14:57: Active query(0): END;
2016-02-13 04:15:07: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:15:07: Active query(0): END;
2016-02-13 04:15:17: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:15:17: Active query(0): END;
2016-02-13 04:15:27: SQLITE_BUSY in CQuery::Execute Stmt: [END;]
2016-02-13 04:15:27: Active query(0): END;
2016-02-13 04:15:37: ERROR: SQLITE: Long running query Stmt: [END;]
2016-02-13 04:15:37: ERROR: Active query(0): END;"

The servers stays in ā€œIndexingā€ state, the client continues with:

2016-02-13 04:19:06: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 04:49:07: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 05:19:08: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 05:49:09: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 06:19:10: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 06:49:11: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 07:19:12: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 07:49:13: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 08:19:14: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 08:49:15: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 09:19:16: Looking for old Sessionsā€¦ 0 sessions
2016-02-13 09:49:17: Looking for old Sessionsā€¦ 0 sessions

and everything is stuck this way.

This seems to happen only on larger servers, and not on first (full or incremental) backup, but on second/third after restarting the client.

Could you create a memory dump of the UrBackupClientBackend.exe process and send that to me (best done with the task manager http://blogs.msdn.com/b/debugger/archive/2009/12/30/what-is-a-dump-and-how-do-i-create-one.aspx )?

This is happening on Linux. I will be doing it as soon as it will happen again.

Assuming you are using the client installer: I havenā€™t saved the debug info for that yet (before stripping it).

Can you perhaps give me steps to reproduce the problem? I could also build one with debug info.

Actually, it happens after one or two backups, without restarting the client. When indexing, that is the error message that shows, and everything hangs there. Restarting the client solves the problem, at least for the first backup. It usually happens when using with big servers or slow ones. It happens both with snapshot modes (datto and lvm) and normal.

When installing on OS X, I get two UrBackup icons showing up in the menu bar. Is that by design? Or is it a bug in the GUI?

Same here

Can you tell me which kind of distribution + Linux kernel version? Thanks!

It surely happened on: Debian 8 (stock kernel: 3.16.0-4-amd64)
Ubuntu 14.04: 3.13.0-77-generic
Centos 7: 3.10.0-327.4.5.el7.x86_64

Yes, I have many combinations here, thatā€™s why Iā€™m doing so many tests.

Hello.

Iā€™m new to this nice thing that called ā€œurbackupā€, and iā€™m starting to love it :smile:

Iā€™v installed this beta version (2.04) and i have question:

when the backup activity (first full backup) stopped suddnely because of unplanned server shutdown (where the urbackup server installed) the backup process wonā€™t start from the last place where it stopped. it will begin a new full backup directoryā€¦ is it right? or there is something to do with it?

thanks in advanceā€¦ Great Work!!