My 2 cents summary of urbackup server options BTRFS vs ZFS vs windows-NTFS

Server
1

GM,
Sorry to bug you guys again but after playing a bit with urbackup and moving from windows to linux-zfs then linux-btrfs… I am trying to summarize my findings and maybe propose to update the urbackup doc a bit as it seems these points cover a lot of repeating questions in the forum…

but first, Can you please confirm that these points are correct?:thinking:
.
SPEED & Stability

  1. urback seems to be much faster on linux than windows *(this might have been a special use-case as other people reported comparable performance)
  2. ZFS-RAIDZ seems much faster than hardware RAID
  3. ZFS-RAIDZ with a cache on SSD (even as small as 16GB) speeds up noticeably urbackup
  4. enabling ZFS dedup kills performance (1/5 speed) takes tons of RAM (2-5GB of RAM per 1TB of disk), and if you max out the RAM, your system will become unresponsive/frozen
  5. At this time, BRTFS RAID5&6 is not stable and would require hardware for RAID5

DEDUP - avoid duplicate copies of the same file/data-block

  1. FILE-DEUP for BTRFS
    urbackup already does file-level dedup with BRTFS including across multiple servers
    (ie: the same file for server B will not be copied a second time if it was already part of a earlier backup from server B and even from an earlier backup of server A)

  2. FILE-DEDUP for ZFS
    urbackup already does file-level dedup with ZFS but only on a per server basis
    (ie: the same file for server B will not be copied a second time if it was already part of a earlier backup from server B HOWEVER it will be copied again if there is a copy from an earlier server A backup, ref:
    Full file backups: what is the purpose? - #2 by orogor )

  3. BLOCK-DEDUP for BTRF
    In a nutshell, setup “Raw copy-on-write file” (COW) and you can run an offline “dedup” script which will try to delete duplicated blocks from one image to the next. But IMHO, this might not be worth the effort as it causes massive I/O and CPU for minimal saves especially with urbackup elegant images-increment-forever. (using beesd after 24h of continuous 100% cpu, saving was 100MB for 2TB)

  4. BLOCK-DEDUP for ZFS
    In a nutshell, setup “Raw copy-on-write file” (COW) and you just nedd to enable dedup=on… but you need tons of RAM and CPU… I don’t know how much you save since it completely killed my PC and I had to power cycle it to recover…

  5. files DEDUP on windows
    urbackup already does file-level dedup with Windows NTFS (using hardlinks) and most likely across multiple clients (same as 5) FILE-DEUP for BTRFS )

  6. BLOCK DEDUP on windows
    It is achievable with NTFS with Windows Server Dedup feature (see below’s reply from Merdona reporting 76% saves!)

Others:

  1. an array of 5+ medium-size SAS drives in RAID5 with raid controler seem to provide a good performance/stability/cost ratio compared to a couple of large SATA disks; however, the power draw is proportional to the number of disks. (see background below for my own setup example)

  2. prefer linux over windows in terms of performance & storage flexibility (raidz, adding/removing disks etc…)

  3. both ZFS and BTRFS have their own strenghts and both are great for urbackup.
    Personally, unless I am missing something,
    14.1) I would prefer ZFS if I could run dedup
    14.2) I would still prefer ZFS without dedup if I didn’t have a large number of clients with a high proportion of identical files (typical home use case).
    14.3) Consequentially I would use BTRFS only if I had a large number of clients with a high proportion of identical files & a RAID5 capable hardware controller (typical work environment where home drives are stored on centralized file servers and builds, hardware, softwares are standardized)

  1. completely uninstall urbackup server including DB then put idents keys back on ubuntu:

echo “saving urbackup keys - run as root”
mkdir /var/tmp/myurbackup-keys
cp /var/urbackup/ident /var/tmp/myurbackup-keys && apt-get remove urbackup-server --purge
echo “if you want to reinstall urbackup, stop the server and copy your idents back”
echo “systemctl start urbackupsrv.service”
echo “cp /var/tmp/myurbackup-keys/ident /var/tmp/myurbackup-keys /var/urbackup”

(ref: Completely reinstall on Debian - #2 by uroni )

  1. Setup linux “backup” partition with zfs raid5 across 6 drives (sdc to sdg) and SSD cache (sda4) with COW, without dedup:

zpool create backup raidz /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg cache /dev/sda4 -m /media/BACKUP
zfs set dedup=off backup
zfs set compression=on backup
echo “backup/images” > /etc/urbackup/dataset
echo “/media/BACKUP/urbackup” > /etc/urbackup/backupfolder
echo “backup/files” > /etc/urbackup/dataset_file

#change USER="root"all the way at the bottom of the file…
vi /etc/default/urbackupsrv
# restart and test
systemctl restart urbackupsrv.service
urbackup_snapshot_helper test
Testing for btrfs…
ERROR: not a btrfs filesystem: /media/BACKUP/urbackup/testA54hj5luZtlorr494
TEST FAILED: Creating test btrfs subvolume failed
Testing for zfs…
ZFS TEST OK
$
(ref: Confused by the doc on zfs copy-on-write (COW) tank - #3 by ffsb)

background:

  • using an array of 6 disks setup as RAID5 (about $120 on ebay for a set of 5 SAS 3TB drives).
  • using an adaptec RAID controler with 512MB of cache (about $70 on ebay).
  • using plain old windows 10 and/or ubuntu workstation with an older AMD APU CPU and 32GB or RAM.
3 Likes
2

You can use it with mdraid . As long as hard disks report disk errors properly this will work out. (The problem being that btrfs can’t error correct because it only sees one disk)

Windows Server has offline Windows Server dedup. It is quite good. You’ll have to store images as vhd, though.

The problem with dedup is that you transform sequential reads/writes into random ones (UrBackup itself already does that compared to e.g. backing stuff up with tar). With those large hard disks which still have about 100 IOPS this can be quite problematic. Hence the cache being quite beneficial.

1 Like
3

about point 11) BLOCK DEDUP on windows

i tested urbackup on windows NTFS with Windows Server Deduplication on a 20 client situation, file and image backup, with amazing result.

Urbackup report a total of 6,28 TB but only 1,5TB are used on filesystem with a deduplication frequency reported by windows of 76%!!!

4

that’s impressive indeed! what about the performance? on my side, the problem was that on windows my file backups hardly had time to finish unlike on ubuntu.

5

acually i noticied no loss of performance.

6

thanks for the feedback I updated my 2 points (perf on windows and dedup on ntfs).

7

I have used the dedup function only in Windows 2012, I also have been highly impressed. Know it has gotten better in 2012R2, 2016 and 2019.

Slightly off topic, but be cautious combing Dedup with BitLocker, if the disk corrupts, very unlikely anything can recover it.

8

In a nutshell, setup “Raw copy-on-write file” (COW) and you can run an offline “dedup” script which will try to delete duplicated blocks from one image to the next. But IMHO, this might not be worth the effort as it causes massive I/O and CPU for minimal saves especially with urbackup elegant images-increment-forever. (using beesd after 24h of continuous 100% cpu, saving was 100MB for 2TB)

@ffsb I am trying to use this setup where I am trying to do block dedup with btrfs. Can someone let me know what is the available offline “dedup” script for btrfs? Any pointers.

1 Like