UrBackup Server/Client/Restore 2.0.0 beta

uroni · January 7, 2016, 8:31pm

Most of the changes are described in the blog:

The Linux command line interfaces have been redesigned since then as well.

Todo

Mac OS X version
UEFI/GPT testing
Update documentation

Known issues

(will be kept up to date)

Pausing/Unpausing does not cause the icon color to change
About page on web interface does not work yet

Compatibility with prior versions

2.x server with 1.4.x client full compatibility (please report issues)
2.x client with 1.4.x server works only in local network mode (not via internet mode)
Older client/server combinations may work but were not tested
1.x restore does not work with 2.x servers (improved login method)

Upgrade process

As always: Replace the executables and the database of the server/client will be updated on first running it. As always downgrading the database version after upgrading it is not possible, so you should backup the old database files especially since this is a beta.

Because of the improved file deduplication and statistics calculation the largest server table has to be completely rebuild. This may take a few hours depending on how many file entries you have. It will show the progress on the web interface but is not usable during the upgrade process.

Linux notes:

The wrapper scripts start_urbackup_server and start_urbackup_client have been removed. Please use the executable directly
The executable has been renamed to urbackupsrv (from urbackup_srv), the client to urbackupclientbackend (from urbackup_client)
There is a new command line interface for the client urbackupclientctl
All the plugins are now statically linked into one executable. This simplifies the compilation, debugging and packaging on Linux

Run the UrBackup server on Linux with e.g. urbackupsrv run --loglevel debug

Downloads

Server: https://limitingfactor.urbackup.org/Server/2.0.0%20beta/
Client: https://limitingfactor.urbackup.org/Client/2.0.0%20beta/
Restore: https://limitingfactor.urbackup.org/Restore/2.0.0/

Misc

@BlackHawkDesign Web API changes from the last version I gave you: I changed the login mechanism to use stronger encryption and all the dates are returned as unix timestamps now

luca91 · January 7, 2016, 11:09pm

Thank you very much for your effort!
I will deploy this version tomorrow and come back as soon as possible with feedback and/or issues.
Let me know if you need any help in updating the documentation.

shedstrom · January 8, 2016, 3:09am

I would like to see the ability to download a specific backup client from the status page of the server. For example, if I create an extra client and configure its settings to be different from other clients I would like the ability to download that client from the server. Also, I would like to be able to download the client under the dropdown for that specific client once it has become an active client. This would help if you needed to reinstall the client for some reason and wanted to ensure the settings for the client were correct.
Thank you for the great program. I like the more modern look and I appreciate the better deduplication.

marveltec · January 14, 2016, 6:41am

Hi,

Ok installed new Urbackup server 2.0.0 beta and left clients on 1.4.10.

first impressions:

web interface is so much nicer has a good clean look to it and the more accurate statistics are a huge step forward. (I will test more and give feedback on possible enhancements when used more)
I did notice however that when trying to access the web interface from i.e. 11 or edge it doesn’t load properly I have to access it through Firefox not sure if this is a bug issue or just localised to my system.
I know in the developer blog its mentioned next beta will support sparse files. is this the reason I’m getting a lot of warnings regarding sparse files while backups run? and does this in anyway effect the validity of the backup?

all and all I’m likening the new look and features I will spend more time with the system and give more solid feedback afterwards in the meantime keep up the great work!!!

uroni · January 14, 2016, 1:22pm

Ok, will have a look at it with IE.

No, there should not be any warnings. Can you please post them?

john3354 · January 14, 2016, 6:11pm

@Uroni, how can I test this in my current environment? The 2.0b client keeps finding the old UrBackup server and attempting to backup to it. Can I lock it to one specific local server?

Any idea when we will be able to download a pre-configured client from the web interface?

john3354 · January 14, 2016, 8:36pm

Even if I kill the other UrBackup server, I cannot get it to discover the 2.0b server that is running. What level of logging do you want @Uroni?

marveltec · January 14, 2016, 8:53pm

Hi Uroni,

The Warning messages in the live log look like this (but there are many of them for all clients)
WARNING Error adding sparse file entry. Could not read metadata from e:\urbackup\client name\160114-2229.hashes\Desktop\Supporting documents\TC Walker\SKMBT_36151202141300.pdf

I’ve also noticed I get errors also they show up in the live log like this
ERROR Cannot copy directory metadata from “e:\urbackup\client name\160113-2345.hashes\Desktop.dir_metadata” to “e:\urbackup\client name\160114-2229.hashes\Desktop.dir_metadata”. - The system cannot find the file specified. (errorcode=2)

None of these two above messages effect the backup status of the client and after a backup completes with the above warning and error message the client is displayed in web status window as ok and having completed a successful backup.

I’m only running over internet backups and this happens across the board for me whether the client Urbackup folder is compressed or not and the server is running on windows server 2012

john3354 · January 14, 2016, 9:02pm

Where are the sources for 2.0b? I like the run-once that you added and I was wanting to see how you handled that in case I wanted to do something similar in other projects.

uroni · January 14, 2016, 9:24pm

Sources are here: https://github.com/uroni/urbackup_backend/tree/dev

Ok, those messages are normal and should appear until you run a full file backup with 2.0 client+server. Nothing to worry about.

See the first FAQ question on how to get it to backup to a different server: https://www.urbackup.org/faq.html

marveltec · January 17, 2016, 1:56pm

Hi Uroni,

Thank you for the feedback i will check on this after a full backup runs. one more question is there a way to download client from web interface like with server 1.4.X?

marveltec · January 21, 2016, 5:40am

Hi Uroni,

Ok so after running a full backup it sorted itself out so i decided to take the plunge and i have upgraded the clients to 2beta aswell after which i restarted all the clients as well as the server now I’m getting failed backups with the following log

Info

16/01/21 00:00
Starting incremental file backup…
Errors

16/01/21 00:03
Constructing of filelist of “Client Name” failed - TIMEOUT(1)
Errors

16/01/21 00:03
Backup had an early error. Deleting partial backup.

this is happening to all my 2beta clients however i have left one 1.4.10 which still works

any ideas as to what could be causing this? just a reminder its over internet backups to a windows 2012 R2 server

LKeeper · January 21, 2016, 10:23am

Windows server 2003 standard edition ServicePack 2

urbackup.log
2016-01-21 13:23:50: ERROR: Loading urlplugin.dll failed
2016-01-21 13:23:50: ERROR: Loading urbackupserver_prevista.dll failed
2016-01-21 13:24:08: WARNING: Error: Unknown action [login]

tarakesh · January 21, 2016, 11:16am

Having a problem while compiling the 2.0.0 Beta client on debian lenny…
Commands: ./configure --enable-headless
configure runs without errors

output of make:

 root@ftp:~/urbackup-client-2.0.0.0# make
make  all-am
make[1]: Entering directory `/root/urbackup-client-2.0.0.0'
g++ -DHAVE_CONFIG_H -I.    -DOS_FUNC_NO_NET -DOS_FUNC_NO_SERVER -DLINUX -DVARDIR='"/usr/local/var"' -g -O2 -MT clientctl/urbackupclientctl-main.o -MD -MP -MF clientctl/.deps/urbackupclientctl-main.Tpo -c -o clientctl/urbackupclientctl-main.o `test -f 'clientctl/main.cpp' || echo './'`clientctl/main.cpp
mv -f clientctl/.deps/urbackupclientctl-main.Tpo clientctl/.deps/urbackupclientctl-main.Po
g++ -DHAVE_CONFIG_H -I.    -DOS_FUNC_NO_NET -DOS_FUNC_NO_SERVER -DLINUX -DVARDIR='"/usr/local/var"' -g -O2 -MT urbackupcommon/urbackupclientctl-os_functions_lin.o -MD -MP -MF urbackupcommon/.deps/urbackupclientctl-os_functions_lin.Tpo -c -o urbackupcommon/urbackupclientctl-os_functions_lin.o `test -f 'urbackupcommon/os_functions_lin.cpp' || echo './'`urbackupcommon/os_functions_lin.cpp
mv -f urbackupcommon/.deps/urbackupclientctl-os_functions_lin.Tpo urbackupcommon/.deps/urbackupclientctl-os_functions_lin.Po
g++ -DHAVE_CONFIG_H -I.    -DOS_FUNC_NO_NET -DOS_FUNC_NO_SERVER -DLINUX -DVARDIR='"/usr/local/var"' -g -O2 -MT urbackupclientctl-stringtools.o -MD -MP -MF .deps/urbackupclientctl-stringtools.Tpo -c -o urbackupclientctl-stringtools.o `test -f 'stringtools.cpp' || echo './'`stringtools.cpp
mv -f .deps/urbackupclientctl-stringtools.Tpo .deps/urbackupclientctl-stringtools.Po
g++ -DHAVE_CONFIG_H -I.    -DOS_FUNC_NO_NET -DOS_FUNC_NO_SERVER -DLINUX -DVARDIR='"/usr/local/var"' -g -O2 -MT clientctl/urbackupclientctl-Connector.o -MD -MP -MF clientctl/.deps/urbackupclientctl-Connector.Tpo -c -o clientctl/urbackupclientctl-Connector.o `test -f 'clientctl/Connector.cpp' || echo './'`clientctl/Connector.cpp
mv -f clientctl/.deps/urbackupclientctl-Connector.Tpo clientctl/.deps/urbackupclientctl-Connector.Po
g++ -DHAVE_CONFIG_H -I.    -DOS_FUNC_NO_NET -DOS_FUNC_NO_SERVER -DLINUX -DVARDIR='"/usr/local/var"' -g -O2 -MT clientctl/urbackupclientctl-tcpstack.o -MD -MP -MF clientctl/.deps/urbackupclientctl-tcpstack.Tpo -c -o clientctl/urbackupclientctl-tcpstack.o `test -f 'clientctl/tcpstack.cpp' || echo './'`clientctl/tcpstack.cpp
mv -f clientctl/.deps/urbackupclientctl-tcpstack.Tpo clientctl/.deps/urbackupclientctl-tcpstack.Po
g++ -DHAVE_CONFIG_H -I.    -DOS_FUNC_NO_NET -DOS_FUNC_NO_SERVER -DLINUX -DVARDIR='"/usr/local/var"' -g -O2 -MT urbackupcommon/urbackupclientctl-escape.o -MD -MP -MF urbackupcommon/.deps/urbackupclientctl-escape.Tpo -c -o urbackupcommon/urbackupclientctl-escape.o `test -f 'urbackupcommon/escape.cpp' || echo './'`urbackupcommon/escape.cpp
mv -f urbackupcommon/.deps/urbackupclientctl-escape.Tpo urbackupcommon/.deps/urbackupclientctl-escape.Po
g++ -DOS_FUNC_NO_NET -DOS_FUNC_NO_SERVER -DLINUX -DVARDIR='"/usr/local/var"' -g -O2   -o urbackupclientctl clientctl/urbackupclientctl-main.o urbackupcommon/urbackupclientctl-os_functions_lin.o urbackupclientctl-stringtools.o clientctl/urbackupclientctl-Connector.o clientctl/urbackupclientctl-tcpstack.o urbackupcommon/urbackupclientctl-escape.o  -lrt  -lrt
g++ -DHAVE_CONFIG_H -I.  -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIE   -pthread -DLINUX -DSTATIC_PLUGIN -DVARDIR='"/usr/local/var"' -g -O2 -MT urbackupclientbackend-AcceptThread.o -MD -MP -MF .deps/urbackupclientbackend-AcceptThread.Tpo -c -o urbackupclientbackend-AcceptThread.o `test -f 'AcceptThread.cpp' || echo './'`AcceptThread.cpp
g++: error: unrecognized command line option ‘-fstack-protector-strong’
make[1]: *** [urbackupclientbackend-AcceptThread.o] Fehler 1
make[1]: Leaving directory `/root/urbackup-client-2.0.0.0'
make: *** [all] Fehler 2

root@ftp:~/urbackup-client-2.0.0.0# g++ --version
g++ (Debian 4.7.2-5) 4.7.2
Copyright © 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

uroni · January 21, 2016, 11:16am

Could you have look at the client side log at C:\Program files\UrBackup\debug.log ?
Could be e.g. a crash. If it crashes without creating a crash dump e.g. here are instructions how to get one anyway: https://msdn.microsoft.com/en-us/library/windows/desktop/bb787181(v=vs.85).aspx?f=255&MSPPError=-2147217396

uroni · January 21, 2016, 11:18am

Thanks for the report. Will be fixed.

marveltec · January 21, 2016, 12:10pm

Hi Uroni,

Thank you for your feedback. Ok before you responded i tried a reinstall of the client 2beta and i think i have found the problem. initially i downloaded an ran the UrBackupUpdate.exe from the 2beta client download links which seemded to have upgraded the client to 2beta and the server web interface was showing that the client was 2beta. (as mentioned before i restarted the client pc)

I have now downloaded and run UrBackup Client 2.0.0 beta.exe on one of the clients that was already upgraded to 2beta and now it is running a backup.

I’m going to try this on all the clients and see if it resolves the issue.

marveltec · January 21, 2016, 2:28pm

Hi Uroni,

Ok i can confirm that backups seem to be working again since running the UrBackup Client 2.0.0 Beta.exe (over the already run UrBackupUpdate.exe) from the client downloads.

Is this the correct way to do the upgrade or should the UrBackupUpdate.exe work?

tarakesh · January 22, 2016, 11:17am

Hi together,
i’m using zfs with compression (gzip-9) as backup space for urbackup.
btrfs does unfortunately not support such high compression. only default gzip is implemented if i read it correct.

Would it be possible to port btrfs hole punching for raw image files to zfs too?
As far as i read zfs does support this…
This could save even more backup space and make the backups as robust as with the btrfs implementation.
activating zfs dedup is a nogo on the backup machine (8GB ram for 6TB backup storage) and would not get me any better ratios (simulated with zdb -S)

uroni · January 22, 2016, 12:20pm

I currently have no ZFS filesystem for testing. Could you change the btrfs calls in https://github.com/uroni/urbackup_backend/blob/next/snapshot_helper/main.cpp to the relevant zfs calls and report back if the raw image backups work on ZFS?