Hi all,
First of all thanks for this software, that I’m traying to use with some key users.
I would check the behavior of my installation UrBackup, because I think that the time taken by the file backup (full or incremental) is too high.
I’ve clients that started their file backup at 9 o’clock am, stucked at 100% with two or three thousand of file in queue at 6 pm. It means that these PC will be shutten down before the end of the backup process.
I tried to follow some suggestions in other thread on this forum, activating the “Cache database type for file entries” option (value: SQLLite), but without obvious effects
Server version: 1.4.10
Client version: 1.4.10
Server type: Windows 2012 R2
Client type: Windows 7
What are the specs of the machine and some details about how it is setup? Processor, memory, how the disks are laid out, etc? Are the backups occurring locally across the LAN or across the internet? How much data are they backing up? Several of those are full file backups, are those their first backups?
The UrBackup server is a ESXi VM running Win2012 R2 (2 vCPU, 4GB RAM). The storage used for UrBackup is a 2TB iSCSI partition, hosted by a QNAP NAS. The server and the NAS are connected by a Gbit HP Procurve Switch.
Clients use mixed connection media: some use 100Mbit wired link, some Gbit wired link, some are connected through WIFI (Aruba Networks IAP-205 - 802.11ac 300Mbit max).
Clients data varies from a minimum of 4GB to a maximum of 60GB (the biggest files are pst files).
And likely that the QNap NAS is choking on all of the work it is doing over iSCSI. The QNap is probably running a RAID5 or RAID6. If the database and the storage are both on that volume then it is choking on disk contention.
When we evaluated UrBackup it was on a bare metal machine that had a RAID5 split in to two volumes. OS on part of it and a storage volume on the rest of it. Performance was pretty horrible. Now that we have moved to an SSD RAID1 for OS (and therefore database) and a RAID6 for storage, performance is massively improved. If they are both on the same volume, it cannot do database operations and storage operations at the same time. You run in to serious disk contention.
How many other VMs are sharing that same physical disk? How many other operations are being performed on the NAS device sharing the iSCSI?
There are a lot of opportunities for disk contention in your setup, unless UrBackup is literally the only thing running on it. Another thing to consider is that the initial full-file backup takes quite a while. After that, incrementals are pretty quick.
Just for info here:
I’m running UrBackup on a fully dedicated machine all Storage locally attached
OS + DB on 4x 15K SAS Raid10
Backups on 4x 7.2K SATA Raid 10
even with that configuration i’m getting close to 30% waiting for IO time in top while writing to the Backup Array…
With a throughput of nearly 150MB/Sec…
Which QNAP are you using? The lower end Models (lower as x59 without Intel CPU) are not capable of giving much throughput in a RAID Configuration. Getting with my TS-412 at home (4x3TB RAID 5) at best 30-40 MB/sec writing…
As I said…
Even with 4 local HDDs in a Adaptec 6805 RAID 10 configuration i’m getting IO bottlenecked…
Too much things that can go wrong in your setup…
As John said: Discs could be saturated from other VMs, Network could be saturated from sharing the ISCSI traffic over the same Interface as the incoming backups, etc…
Have you enabled the following options?
Temporary files as file backup buffer
Temporary files as image backup buffer:
With that all files are stored in a temp directory before getting written to the backup storage. If all files are copied the progress is sitting at 100% with x files in queue. In that moment the client can go offline without problems and the server can finish the copy to the backup storage in the background.
If you did not enable this settings and you plan to do so keep in mind to scale the free space in temp according to the BIGGEST file you gonna backup times the number of backup clients that are running backups at the same time + 10-15% reserve
If that is still to slow you should consider to lower the number of simultanious backups.
PS Why are you running the backup server as a VM and not on the QNAP Directly?
There’s your problem. That NAS is made for storing movies, not running a virtualization workload. It doesn’t have the horsepower to do what you want to do. You are not going to get the bandwidth that you need, and if you are running any other VMs over iSCSI at the same time it will only get worse.
The cheap way out of that would be to buy some more drives and put your storage on bare metal. If you want to keep with iSCSI, you are going to need a much better NAS. Try a few large copies to that iSCSI share and I bet you would average less than 20MB/s.
Another thing to keep in mind is that the hard link performance of the Windows file system NTFS can be worse than that of other file systems. This matters especially if you run a lot of full backups, disabled the use of symlinks for incremental backups or have a lot of clients with the same files.
The hard linking imposes a database style workload on the storage for which RAID5/RAID6 + hard disks are not suited performance wise.
That said, you should use system tools to confirm/find the bottleneck. That is if it is random IO, IO, CPU, network, your server or your NAS.
I’m checking if I can use the remaining ESXi internal datastore as additional space to locally store the backup partitions (also adding a couple of disks - I need at least 2TB to manage my clients).
In the meanwhile, after the first one or two days, the backup process seems to be faster, and clients are finish backup after maximum 3 hours (194 minutes the slower one).
