Server 2.4.2 beta/Client 2.4.0 beta

Major changes with server 2.4.x beta

  • IPv6 support (sponsored by Tuxis Internet Engineering )
  • ZSTD transport compression
  • Parallel hashing improvements

Changes with server 2.4.2 beta

  • Fix FreeBSD compile issue

Changes with server 2.4.1 beta

  • Fix polling with ipv6 socket on Linux in ServiceAcceptor
  • Fix report script cache reload
  • Fixup backup folder location correctly (trailing slash)
  • Fix IPv6 on FreeBSD (not compile tested yet)

Changes with server 2.4.0 beta

  • Allow setting group id while adding new clients
  • Postpone metadata application of files that are in queue before reordered downloads (because of missing file in previous backup)
  • Allow ldap admin users access to all backups
  • Set file backup to incomplete during startup recovery
  • New setting to make backup dirs optional per default
  • Cleanup one image backup of each client at a time when freeing space (like with file backups)
  • Catch and log info about C++ exceptions
  • Improve parallel hash logging and properly set eof when finished
  • Do not retry getting parallel hash after error
  • Check if VHD file was opened successfully before using it (Fixes Windows crash)

Major changes with client 2.4.x beta

  • IPv6 support (sponsored by Tuxis Internet Engineering )
  • ZSTD transport compression
  • HTTPS proxy support (only with SChannel on Windows for now)
  • Parallel hashing improvements

Changes with client 2.4.1 beta

  • Fix FreeBSD compile issue

Changes with client 2.4.0 beta

  • Reset reconnect tries if more than 30min between reconnects
  • Only set reconnect_tries to 50 if it wasn’t set higher previously
  • Tool to read volume and write zeros to blocks which cannot be read
  • Set socket window size to 3MiB send buffer and 128KiB recv buffer for client on Windows
  • Fix incr image bit clashing with full file backup bit
  • Remove result writing via pipe fixing some async indexing hang-ups
  • Fix search for snapshot if VSS path is the root of a volume
  • Fix parallel hashing with VSS components
  • Hard code not following default wine z: symlink to root of fs on non-Windows
  • Functionality to interrupt parallel hash loading threads on clients
  • Don’t parallel hash files smaller than 2048 bytes
  • Add to db if there is a sub-directory because that one can be deleted and should then be removed from the db (Linux/MacOS)
  • Do not quit indexing with “no backup dirs” if components are selected for backup

Changes with Restore CD 2.2.2 beta

  • Fix GPT restore with restore disk being (slightly) smaller

Upgrade process

As always: Replace the executables (via the installers) and the database of the server/client will be updated on first running it.

Place the files from the update directory into C:\Program Files\UrBackupServer\urbackup or /var/urbackup to auto-update clients. Disable Download client from update server in the server settings to prevent the server from downloading the current version.

On Linux e.g. with this update script: https://github.com/ptempier/get_urbackupclient/blob/master/updateclient.sh

Downgrade process (server)

Stop the UrBackup server, restore C:\Program Files\UrBackupServer\urbackup or /var/urbackup from a backup before upgrade and then install the previous version over the beta release.

Downloads

I thought I’d get a jump start working on the FreeBSD port updates based on these betas. Both the server and the client are having errors (on 12.0-RELEASE-p4) in compiling urbackupcommon/fileclient/FileClient.cpp:

urbackupcommon/fileclient/FileClient.cpp:595:15: error: no member named 'sin_add' in 'sockaddr_in'
                                        addr_udp.sin_add = udpsocks[i].broadcast_addr;
                                        ~~~~~~~~ ^
urbackupcommon/fileclient/FileClient.cpp:647:10: error: no matching function for call to 'setsockopt'
                                                if(setsockopt(udpsocks[i], IPPROTO_IP, IP_ONESBCAST, &optval, si...
                                                   ^~~~~~~~~~
/usr/include/sys/socket.h:689:5: note: candidate function not viable: no known conversion from
      'std::__1::__vector_base<FileClient::SSocket, std::__1::allocator<FileClient::SSocket> >::value_type'
      (aka 'FileClient::SSocket') to 'int' for 1st argument
int     setsockopt(int, int, int, const void *, socklen_t);
        ^
urbackupcommon/fileclient/FileClient.cpp:690:7: error: use of undeclared identifier 'optval'
                                                optval = 1;
                                                ^
urbackupcommon/fileclient/FileClient.cpp:691:62: error: use of undeclared identifier 'optval'
                                                if (setsockopt(udpsocks[i], IPPROTO_IP, IP_ONESBCAST, &optval, s...

Re: the first error on line 595, I think the assignment should be to “addr_udp.sin_addr.s_addr” (as in line 597) instead of to “addr_udp.sin_add”. Looking at /usr/local/include/netinet/in.h (included by urbackupcommon/fileclients/socket_header.h), netinet/in.h defines a “sockaddr_in” struct (of which addr_udp is an instance). This struct has a sin_addr member, which is itself a stuct having an s_addr member.

You are right, will look into it.

Hi uroni, can you comment on the ZSTD transport compression? Will it offer faster or less CPU intensive backups?

Thank you, Tuxis Internet Engineering, and repeated thanks, uroni, for maintaining and improving this program. Too bad Apple is acting like old Microsoft and changing the OS to kill competing applications.

Tried this beta at one of my offices that has dual stack v4/v6 working. Environment is Active Directory, Windows Server 2008r2 server, Windows 7 Pro clients. Had two sets display “Server Authentication Failed” messages in the status web page, this was after the new 2.4.0 client had auto-installed, but others without issue so far.

For those two clients I deleted the three *_idents.txt files (there is no overkill…) and restarted. Also restarted the UrBackupWinServer service because I’m impatient. The idents files rebuilt successfully and clients now show online as expected.

Just for a complete report, the office server is also a backup client to another office. I accidentally upgraded it to 2.4.0 beta client when installing the server beta. Seems to work fine with the Internet-connected Server 2.3.8 in a nearby city. Really nice having betas that are production quality as opposed to many other companies with production releases that are beta quality at best.

Haven’t tried an IPv6-only client yet - that will come this weekend when I can disable IPv4 on a few clients without endangering office operations.

That issue is quite critical. I wonder if it would have resolved itself with client/server restarts or with time.

Haven’t tested with IPv4 or IPv6 only yet. So it probably won’t work, i.e. it will fail to start because it can’t bind to a IPv4 or IPv6 interface. Does anyone know how this is usually handled, do apps just ignore if IPv4 or IPv6 fails as long as one of them works?

No it is much worse :wink: Zstandard - Real-time data compression algorithm No multi-threading support on the sending side yet – I’ll look into that in the future. It might need to do much more send buffering (in the order of ~100MB or so)

Wow! For comparison, what was/is currently used on the previous versions for compression? If it is Zlib… that is GREAT!

You usually try to bind ::, which includes IPv4 and IPv6. It’s almost impossible to have a machine nowadays that doesn’t allow you to do so. If you configure explicit addresses, then you fail if you can’t bind to those addresses.

Yay!

I see urbackup listening on ipv6 addresses, but I get no response from [::1]:55414 or one of the external addresses. [This is on Linux btw]

If anyone has a disk with damaged sectors, I’d be very interested if the “disk scrubbing” tool works (at "C:\Program Files\UrBackup\scrub_disk.bat" )

Server 2.4.1 beta (see edited first post) fixes the ipv6 issues on Linux.

Previous issue with server authentication seems to be either solved or intermittent enough not to be a problem. Only saw one instance recently which went away on its own after a while.

Latest annoyance is timeout (2) errors on incremental file backups for three clients, all with working dual-stack network connections. Upgraded server to 2.4.1 and manually tried full backups. One completed successfully that was run by itself, two failed that were started at the same time. I’m busy with other priorities so no time for better testing right now, sorry. All had been working under latest stable release but have not had successful incrementals under 2.4. Rest of office seems happy.

Environment is Windows Server 2008r2 and Windows 7 Pro clients.

I am having a problem with the client, it will not binding to the UDP port. If I change nothing other than down grade to the current release version it works fine. There is nothing else on the port when I try version 2.4.0.

Compilation failed with 2.4.1 on FreeNAS. Hope this log helps :

– urbackupcommon/urbackupsrv-TreeHash.o —
mv -f urbackupcommon/.deps/urbackupsrv-TreeHash.Tpo urbackupcommon/.deps/urbackupsrv-TreeHash.Po
— urbackupcommon/fileclient/urbackupsrv-FileClientChunked.o —
c++ -DHAVE_CONFIG_H -I. -I/usr/local/include -I/usr/local/include -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIE -DSQLITE_PREPARE_RETRIES=5 -DNDEBUG -I/usr/include -I/usr/include -D_THREAD_SAFE -pthread -DLINUX -DSTATIC_PLUGIN -DVARDIR=’"/usr/local/var"’ -DBINDIR=’"/usr/local/bin"’ -DDATADIR=’"/usr/local/share"’ -DNO_ZSTD_COMPRESSION -g -O2 -MT urbackupcommon/fileclient/urbackupsrv-FileClientChunked.o -MD -MP -MF urbackupcommon/fileclient/.deps/urbackupsrv-FileClientChunked.Tpo -c -o urbackupcommon/fileclient/urbackupsrv-FileClientChunked.o test -f 'urbackupcommon/fileclient/FileClientChunked.cpp' || echo './'urbackupcommon/fileclient/FileClientChunked.cpp
— urbackupcommon/fileclient/urbackupsrv-FileClient.o —
1 error generated.
*** [urbackupcommon/fileclient/urbackupsrv-FileClient.o] Error code 1

make[1]: stopped in /urbackup-server-2.4.1.0
— urbackupcommon/fileclient/urbackupsrv-tcpstack.o —
mv -f urbackupcommon/fileclient/.deps/urbackupsrv-tcpstack.Tpo urbackupcommon/fileclient/.deps/urbackupsrv-tcpstack.Po
— urbackupcommon/fileclient/urbackupsrv-FileClientChunked.o —
mv -f urbackupcommon/fileclient/.deps/urbackupsrv-FileClientChunked.Tpo urbackupcommon/fileclient/.deps/urbackupsrv-FileClientChunked.Po
— sqlite/urbackupsrv-sqlite3.o —
mv -f sqlite/.deps/urbackupsrv-sqlite3.Tpo sqlite/.deps/urbackupsrv-sqlite3.Po
1 error

make[1]: stopped in /urbackup-server-2.4.1.0
*** [install-recursive] Error code 1

make: stopped in /urbackup-server-2.4.1.0
1 error

make: stopped in /urbackup-server-2.4.1.0
root@Urbackup:/urbackup-server-2.4.1.0 #

1 Like

@freebsd users:

Fixed the server compilation with 2.4.2 and it seems to run.

The fixed client is here http://beta.urbackup.org/Client/2.4.1%20beta/urbackup-client-2.4.1.0.tar.gz but it fails to run in my FreeNAS jail (fails to join IPv6 multicast group). I’ve even looked at the FreeBSD kernel code and my guess is it doesn’t work because my jail doesn’t have a network interface with an assigned IPv6 address.
Going forwards I guess, I’d have to make the IPv6 stuff optional (so it does startup if IPv6 stuff fails) at least on FreeBSD.

Compiled 2.4.2 server fine on the latest stable FreeNAS. Restarted. I cant reach the GUI with the IPv4 Address. Back to 2.3.8 and everything is working ok. Any clue ?