File backup really slow, image backup normal


#1

Hi all.
As in thread, I’m having problem with speed of file backups. It’s 50-300 Kbit/s, while image backups are 5-10 Mbit/s for each client.

Server - 2.3.7 (2.3.8 installed today, but nothing changed) on Debian 9.8 as VM in Proxmox 5.3.
CPU - all 6 cores of i5-9600K (usage 2-5%)
RAM - 4 GB (“free -m” shows ~400 MB used, ~100MB free, rest as buff/cache)
Storage is OpenMediaVault shares mounted vis nfs.

Clients (20 PCs, a few laptops) - 2.3.4 on Windows 7 and 10

Backups are done via LAN, but clients are behind NAT (VLANs made by pfsense).

I tried what I’ve found in other threads (background priority disabled, backups not stopped by clients).

Thanks in advance for your help.


#2

Did you run the image backups in the same time window as the file backups? Just a wild guess, but maybe there were some other traffic-heavy tasks running at the same time as your file backups (e.g. updates).


#3

I chose to exempt the server’s data storage directory from real-time anti-virus scans.
Now, file backups move quickly.


#4

There is no active AV on debian, on OMV, or on pfsense.
I will leave some PCs on for 24 hours and see what happens outside of normal working hours (both file anf image backups was made ~7:00-16:00).


#5

When I left one PC turned on while all other computers worked as always (7:00-15:00 or 8:00-16:00), backup was done at little more than 1 Mbit/s. It’s not fault of speed of the connection - simultaneous image backups speed is still fully acceptable (theoretically, link speed is 1 Gbit/s). Maybe it’s problem with server performance? Any ideas what (and how) can I check?


#6

I would assume this is due to overhead with running the UrBackup storage over NFS. File creation (and possibly symlinks too?) over network shares can take significantly longer than local disks. The server is spending far more time waiting for the storage server to confirm creation of new files than it spends actually sending data into them. Solutions:

  • Enable Temporary files as file backup buffer on the server in Settings -> Advanced. This will store the backup entirely within the UrBackup server VM (/tmp to be specific) while the client sends it, then move it to storage after. Of course the VM will need sufficient space on the primary disk or wherever /tmp is backed. This is also a good test to see if I am correct.

  • Switch to iSCSI instead of NFS.


#7

That was the problem I suppose. After switching to iSCSI speed of backups is finally normal (~40 Mbit/s per client). Thanks a lot :smiley:
I will confirm if everything gone fine when all backups will be done.