Urbackupclient uses 1 core during backups on local host

[DigitalLabs]

Hi everyone!

Just needing some guidance on what I can do to resolve this issue.

For context, this NAS is running on a quad-core i5, openmediavault, 12GB of RAM, and two 1TB HDDs (just single ZFS volumes). The idea is that I have a primary share on the first 1TB, which through urbackup, hashes and then stores backups on the secondary 1TB.

During this testing, I have run into some weird performance issues, mainly the urbackupclient seemingly maxing out 1 single thread, as per top and htop below:

I’m just wondering whether the urbackupclient using just a single core bottlenecks the overall transfer speed, which is currently transferring at about 100-80mbit/s.

I’ve tried messing around with RAW and hashed, number of parallel file download threads, compression, and parallel hash threads in the attempt to improve transfer performance. Unfortunately, this hasn’t really worked, so my best hunch is the lack of multi-core utilisation.

If anyone has any idea how I can rectify this, that would be fantastic. Urbackup and Infscape are great products, and having any information here would certainly help others in the future

Hope everyone is doing well!

Thanks!

Server: UrBackup 2.5.26

UrBackup Client Controller v2.5.20.0

[anon16079863]

I’m having the EXACT same behavior on a Raspberry pi 4.
One cpu core is at 100% and the rest totally idle.

First I thought the 100% meant all cores were fully utilized, only to realize when switching from top to itop that ONE core is only utilized.
25% of the processor speed means transfer speeds of avarege 40mbit on a local > local drive witch means DAYS to do a TB filebackup. I ended up just “dealing with it” like a lot of stuff with urbackup (sadly but a bit understandable, only a few ppl “working” on urbackup if I understand it correctly) because of lack of info on how to solve things.

Maybe it has to do with server and client on same linux machine?

[DigitalLabs]

Hi @anon16079863,

Thanks for the reply, glad I’m not the only one who has a similar problem.

I think it comes down to a matter of priority, I’m sure if more and more people start having this issue, then it’ll get a greater amount of attention.

[uroni]

Please re-test with the latest version and try disabling encryption & compression for the client.

[DigitalLabs]

Hi @uroni,

Thanks for your response.

I’ll get that arranged and let you know how I go.

Thanks

[anon16079863]

I ran this with encryption and compression disabled on server version 2.5.26 and client version 2.5.20.

Ignore the full swapfile, its not being used as seen in picture 2.

I noticed there is a newer client version available (2.5.21) online, is this with a patch you want to try out?

Edit.
I just re-read op post and realized we are using the EXACT same clients. Will be back when new client is downloaded and backups are run.

[anon16079863]

Ok, found the changelogs about the client and saw that disabling encryption and compression might not have been turned off before. instealled the new client but check the version it reports, same on the urbackup gui, states version 2.5.20.

bedna@piServer:~ $ TF=$(mktemp) && wget "https://hndl.urbackup.org/Client/2.5.21/UrBackup%20Client%20Linux%202.5.21.sh" -O $TF && sudo sh $TF; rm -f $TF
--2022-11-10 13:49:59--  https://hndl.urbackup.org/Client/2.5.21/UrBackup%20Client%20Linux%202.5.21.sh
Resolving hndl.urbackup.org (hndl.urbackup.org)... 198.58.118.162, 2600:3c00::f03c:91ff:fec8:ecf6
Connecting to hndl.urbackup.org (hndl.urbackup.org)|198.58.118.162|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 25448161 (24M) [text/x-sh]
Saving to: ‘/tmp/tmp.onPshY50KX’

/tmp/tmp.onPshY50KX                     100%[===============================================================================>]  24.27M   137KB/s    in 2m 33s

2022-11-10 13:52:32 (163 KB/s) - ‘/tmp/tmp.onPshY50KX’ saved [25448161/25448161]

Verifying archive integrity... All good.
Uncompressing UrBackup Client Installer for Linux  100%
Installation of UrBackup Client 2.5.20 to /usr/local ... Proceed ? [Y/n]
y
Uncompressing install data...
Detected Debian \(derivative\) system
Detected systemd
Detected architecture aarch64-linux-android
Installing systemd unit...
Restarting UrBackup Client service...
Successfully started client service. Installation complete.
bedna@piServer:~ $ sudo systemctl status urbackupclientbackend.service
● urbackupclientbackend.service - UrBackup Client backend
     Loaded: loaded (/etc/systemd/system/urbackupclientbackend.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-11-10 13:52:48 CET; 38s ago
   Main PID: 13846 (urbackupclientb)
      Tasks: 11 (limit: 8986)
        CPU: 266ms
     CGroup: /system.slice/urbackupclientbackend.service
             └─13846 /usr/local/sbin/urbackupclientbackend --config /etc/default/urbackupclient --no-consoletime

Nov 10 13:52:48 piServer systemd[1]: urbackupclientbackend.service: Consumed 51min 16.383s CPU time.
Nov 10 13:52:48 piServer systemd[1]: Started UrBackup Client backend.

I’ll still try to catch it when backup is running natively and see if anything has changed.

[anon16079863]

I watched it do a backup and there is clearly something different now. All cores seems to be utilized.
I didn’t have much time to observe since it was only a small amount that needed to be backed up.
I’m going to be doing some big changes on some of the drives in the near future and when that backup runs I will observe and return with a more detailed analysis.

[DigitalLabs]

Hi @anon16079863,

I updated everything as per @uroni’s suggestion, and I believe the recent changes have resolved the issues we were both experiencing.

Was able to get roughly 1.4Gbit/s without encryption and compression while running Urbackup server in docker

@uroni, appreciate your help and support.

[anon16079863]

@DigitalLabs agree.
I did not get a chance to make a screen capture, but my speeds are now up to around 400Mbit/s witch is about 10times the speed I had before.