'Files in Queue' has very large numbers that decrease very slowly


#1

I have 3 separate urbackup servers that are running Ubuntu 16.04.05. 2 of the 3 servers are backing up fast and building up the number of files in the queue then it take many hours for the queue to empty.

The servers are attached to a datastore and each server is running a 2.5 TB virtual storage presented for kernel and storage.

The 1 server that runs fast never builds up the queue more than a couple thousand and empties quickly when it does have file in a queue.

The servers are all VMs on VMware 6.5.

Any advice on how to make the queues flush out faster would be much appreciated.


#2

I get that on servers with a lot of small files (> 500gb mails/svn servers).
Maybe it s worth to have a look at the stored files and setup some exceptions.

I also get that on the aws efs storage, because it needs some specific usage pattern to get some performances ou of it (parallel access with large blocks)


#3

Most of this still applies: http://blog.urbackup.org/177/performance-considerations-for-larger-urbackup-server-instances


#4

Having an SSD for the Program Files/UrBackup directory and another 256GB SSD for urbackup_tmp (Settings: Server: Nondefault temporary file directory) keeps the Files in queue close to zero at my sites.


#5

It’s interesting that you suggest keeping the urbackup_tmp_files on SSD storage. I keep my database files (/var/urbackup under Linux) on SSD storage, as recommended, but I never thought about urbackup_tmp_files.

Do you see a very big difference when you store this directory on SSD storage? Does anyone else have any experience with this change?