UrBackup Server 2.0.10 (updated)/Client 2.0.9 beta (updated)

Most of the changes are described in the blog:

The Linux command line interfaces have been redesigned since then as well.

This version now includes a Mac OS X client and a portable Linux client (for x86/x86_64/ARMv6/ARM64).
The portable Linux client doesn’t have C+±exception support (yet). In my tests everything was functional but this is something that must be resolved before releasing a non-beta version.

Changes with client 2.0.9 beta

  • Fixed bug where renamed directory was not deleted from index cache
  • Added system error code client logging for FILE_DOESNT_EXIST error

Changes with server 2.0.10 beta

  • Fix deletion issue with btrfs storage mode
  • Pause retrying backup for 5min after backup fails without exponential backoff
  • Improved error messages
  • Handle read errors from file list

Changes with server 2.0.9/client 2.0.8 beta

  • Improved file index creation (Create files index by creating a copy and then operating on the copy)
  • Define fallocate constants if not defined (Linux build fix)
  • Try different adduser syntax (for CentOS)
  • Updated SQLite to newest version
  • Do not start sparse extent with last block when tree-hashing file patch
  • Improved backup hooks (scipts) on client and server and documented them
  • Fixed Windows app verifier error on client name change
  • Fixed bug where sparse file extents were not skipped with tree-hashing causing it to calculate wrong hashes on the server
  • Fixed tree diff algorithm causing too large diffs in some cases (too large incremental file backups)
  • Use posix_spawn instead of fork on Linux for the script output backup as the fork causes files to stay open too long (and then snapshots cannot be deleted because there are open files on the snapshot)
  • Do not retry opening hash file for reading
  • Work-around lsblk --paths not working in LVM snapshot script
  • Use sha512 without sparse handling for script output hashing for now
  • Removed hash verification from client (should at least double client side hash speed)

Todo

  • UEFI/GPT testing
  • Document and fix LDAP/AD login

Compatibility with prior versions

  • 2.x server with 1.4.x client full compatibility (please report issues)
  • 2.x client with 1.4.x server works only in local network mode (not via internet mode)
  • Older client/server combinations may work but were not tested
  • 1.x restore does not work with 2.x servers (improved login method)

Upgrade process

As always: Replace the executables and the database of the server/client will be updated on first running it. As always downgrading the database version after upgrading it is not possible, so you should backup the old database files especially since this is a beta.

Because of the improved file deduplication and statistics calculation the largest server table has to be completely rebuild. This may take a few hours depending on how many file entries you have. It will show the progress on the web interface but is not usable during the upgrade process.

Linux notes:

  • The wrapper scripts start_urbackup_server and start_urbackup_client have been removed. Please use the executable directly
  • The executable has been renamed to urbackupsrv (from urbackup_srv), the client to urbackupclientbackend (from urbackup_client)
  • There is a new command line interface for the client urbackupclientctl
  • All the plugins are now statically linked into one executable. This simplifies the compilation, debugging and packaging on Linux

Run the UrBackup server on Linux with e.g. urbackupsrv run --loglevel debug

Downloads

When you say to upgrade (just replace the executables), that means I should just run the EXE file and let it finish right? I don’t actually need to extract EXE files and copy over, etc.

Latest crash with 2.0.9 Server
urbackupsrv cleanup -a 100%
results after some Seconds of cleanup in:

2016-04-06 12:53:51: Changed file index entry from 527261 to 1240089 (next)
2016-04-06 12:53:51: Changed file index entry from 527262 to 1240090 (next)
2016-04-06 12:53:51: Changed file index entry from 527263 to 1240091 (next)
2016-04-06 12:53:51: Changed file index entry from 527264 to 1240092 (next)
2016-04-06 12:53:51: Changed file index entry from 527265 to 1240093 (next)
2016-04-06 12:53:51: Changed file index entry from 527267 to 1240094 (next)
2016-04-06 12:53:51: Changed file index entry from 527268 to 1240095 (next)
2016-04-06 12:53:51: Changed file index entry from 527269 to 1240096 (next)
2016-04-06 12:53:51: Changed file index entry from 527277 to 1240097 (next)
2016-04-06 12:53:51: Changed file index entry from 527279 to 1240098 (next)
2016-04-06 12:53:51: Changed file index entry from 527281 to 1240100 (next)
2016-04-06 12:53:51: Changed file index entry from 527282 to 1240101 (next)
2016-04-06 12:53:51: Changed file index entry from 527286 to 1240102 (next)
2016-04-06 12:53:51: Changed file index entry from 527289 to 1240103 (next)
*** Error in `urbackupsrv': corrupted double-linked list: 0x0000000002b28ed0 ***
Abgebrochen

Could you run it in gdb?

Specifically

gdb --args urbackupsrv cleanup -a 100%
run

once it crashes:

bt

will try on next crash. atm i solved it by recreating the backup directory
do i need to have the debug symbols installed for this?

If you are using the debian package installing the debug symbols + using gdb from debian unstable will show the source locations. Without debug symbols I should still be able to reconstruct the source locations from gdb output.

If you were using the debian package: Perhaps the previous crash showed up with more info in dmesg?

nope didn’t show up in dmesg unfortunatly…

and the next:

Errors07.04.16 15:12 Cannot get file type in ::deleteFilesInSnapshot. No such file or directory (errorcode=2)
Errors07.04.16 15:12 Deleting files in snapshot failed (Server error)
Errors07.04.16 15:12 Backup had an early error. Deleting partial backup.

Persists even if backup storage is beeing deleted and database is cleared.
Initial Full Backup runs. First Incremental results in that error.
Server runs Debian 8, urbackup server 2.0.9 with BTRFS Backup Storage

In “backup status” the Client Version still shows 2.0.7. I am running 2.0.8 client on Win7.

Same issue for me, also on Debian 8 / URB 2.0.9 / WIN client 2.0.8 / BTRFS storage :

07/04/16 17:19 INFO Starting incremental file backup... 07/04/16 17:19 DEBUG SVTEST4: Doing backup with hashes... 07/04/16 17:19 DEBUG SVTEST4: Connecting for filelist... 07/04/16 17:19 DEBUG SVTEST4: Waiting for filelist 07/04/16 17:20 INFO Scanning for changed hard links on volume of "C"... 07/04/16 17:20 INFO Indexing of "C" done. 3031 filesystem lookups 30579 db lookups and 56 db updates 07/04/16 17:20 INFO Scanning for changed hard links on volume of "S"... 07/04/16 17:20 INFO Indexing of "S" done. 5 filesystem lookups 65 db lookups and 5 db updates 07/04/16 17:20 DEBUG SVTEST4: Doing backup with hashed transfer... 07/04/16 17:20 INFO SVTEST4: Loading file list... 07/04/16 17:20 DEBUG SVTEST4 Starting incremental backup... 07/04/16 17:20 INFO SVTEST4: Calculating file tree differences... 07/04/16 17:20 INFO SVTEST4: Creating snapshot... 07/04/16 17:20 INFO SVTEST4: Deleting files in snapshot... (195) 07/04/16 17:20 ERROR Cannot get file type in ::deleteFilesInSnapshot. No such file or directory (errorcode=2) 07/04/16 17:20 ERROR Deleting files in snapshot failed (Server error) 07/04/16 17:20 ERROR Backup had an early error. Deleting partial backup.

The first full file backup runs fine but the next incremental fails with the error above.

Linked new 2.0.10 which should fix the btrfs issue.

works like a charm…
its getting better and better :slight_smile:

I have had no issues as of v2.0.8 beta and higher

The only internet client I have tried so far keeps failing out on an .eps file. Is it possible that file type is causing the issue?

File type should not matter. Could you check if the file is encrypted (EFS) or blocked by a on-demand virus scanner?

Okay it works fine now with 2.0.10, thks for your great work !

Neither of those. The file system is not encrypted and there is no virus scanner installed on the file server.

Could you post the exact error message, perhaps including a client debug log? Thanks!

Debian 8 Server 2.0.10 and Windows 7 Client 2.0.8 gives me following error (incremental) on a client.

Level	Time	Message
Errors
09/04/16 10:00
Error getting complete file "ya3Q9D7XGAOc6cY4vNQ2|C/Users/Tina/AppData/Local/Microsoft/Windows Live/Contacts/Default/15.5/DBStore/Backup/new/edb0051D.log" from Tina PC. Errorcode: FILE_DOESNT_EXIST (3)
Errors
09/04/16 10:00
Client Tina PC went offline.
Errors
09/04/16 10:02
Error writing file metadata -2
Errors
09/04/16 10:02
Error writing file metadata -2
Errors
09/04/16 10:02
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error writing file metadata -2
Errors
09/04/16 10:03
Error getting file metadata. Errorcode: TIMEOUT (2)
Errors
09/04/16 10:04
Backup failed