I’ve been troubleshooting my slow image backups as of late. I’ve determined that the main cause for the slow performance is the HDD I am using for backups: A “Western Digital Technologies, Inc. Elements Desktop”-USB3.0 Enclosure containing a 10TB WDC WD101EMAZ-11G7DA0.
=== START OF INFORMATION SECTION === Device Model: WDC WD101EMAZ-11G7DA0 Serial Number: VCGRK2RP LU WWN Device Id: 5 000cca 0b0ca404f Firmware Version: 81.00A81 User Capacity: 10,000,831,348,736 bytes [10.0 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5400 rpm Form Factor: 3.5 inches Device is: Not in smartctl database [for details use: -P showall] ATA Version is: ACS-2, ATA8-ACS T13/1699-D revision 4 SATA Version is: SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Mon Sep 18 14:18:51 2023 CEST SMART support is: Available - device has SMART capability. SMART support is: Enabled
I am running UrBackup 2.5.31 on a debian host via docker. The drive is formatted in BTRFS and I’m making heavy use of the BTRFS features in UrBackup. The drive is capable of around 160MB/s of throughput, if I transfer a single large file via rsync, however I did see some fluctuation in the speeds the drive delivered… Perhaps something the USB-SATA controller screws up?
Anyways, I managed to improve backup performance by delete all old snapshots and defragging the client volume on the backup drive (
btrfs filesystem defrag -r /backups/client), this allowed the backup to finish with a speed of around 400mbit/s. However, while trying to restore the backup to an SSD drive right now, the speed hovers around 10-20MB/s, so the restore of 170GB of data would take around 5h.
Is there any way I can improve the speed of this process? It seems like the limiting factor is again the drive:
iostat -xdch -p /dev/sdf1
avg-cpu: %user %nice %system %iowait %steal %idle 0.9% 0.1% 14.8% 16.4% 0.0% 67.8% r/s rkB/s rrqm/s %rrqm r_await rareq-sz Device 108.50 27.2M 2.00 1.8% 13.48 256.6k sdf1 w/s wkB/s wrqm/s %wrqm w_await wareq-sz Device 0.00 0.0k 0.00 0.0% 0.00 0.0k sdf1 d/s dkB/s drqm/s %drqm d_await dareq-sz Device 0.00 0.0k 0.00 0.0% 0.00 0.0k sdf1 f/s f_await aqu-sz %util Device 0.00 0.00 1.46 82.6% sdf1
This is how the drive is mounted: (rw,noatime,compress=zstd:3,space_cache=v2,subvolid=5,subvol=/)