Since time ago I always have been using Linux with Debian distros to install Urbackup Server with BTRFS to support snapshotting and their fabulous beneficts. From UrBackup perspective I have seen that BTRFS is almost perfect… but from OS perspective BTRFS is quite annoying and require more management and expertise in some cases (i.e. like RAID degradations or filesystem problems)…
So I am thinking about start trying TrueNAS which have very well integrated ZFS filesystem which as far as I readed is better than BTRFS in every aspect. Other option is Open ZFS in Linux too.
I will appreciate anyone that can share his TrueNAS or OpenZFS experience with UrBackup.
I don’t have any experience with TrueNAS but I was running FreeBSD for years, including UrBackup servers. ZFS integration on FreeBSD is really awesome, there is nothing like that on Linux side… OpenSUSE+btrfs perhaps.
But today while I have a great sentiment for FreeBSD, for many reasons I’m choosing Linux every time. The bottom line is FreeBSD project somehow have lost the attention of developers over the years. It used to be other way round but now you will hardly find any innovation on FreeBSD. Even iXsystems, a core FreeBSD company, is investing in Linux with TrueNAS SCALE.
OpenZFS on Linux is stable and works like a charm. I’m running a bunch of UrBackup+ZFS servers on Ubuntu and didn’t have any issues. Including pet servers on Raspberry Pi! Don’t believe in all that ‘zfs requires tons of ECC RAM’ nonsense, it was debunked long time ago.
One important thing to keep in mind is that some Linux distributions are really anal about licensing and don’t give a f* about randomly breaking your zfs modules on kernel updates. So you better pick distribution with explicit support for ZFS or be careful with updates.
As to btrfs vs ZFS it looks like you already know the difference For me it’s always ZFS, but if I had to go with btrfs then only on SUSE where it’s a first class filesystem.
I didn´t know that TrueNAS Scale was Linux Debian based. This could be a great start point thanks to the fabulous ZFS management and web UI integration with TrueNAS… and with containers support. Deffinitively I´m going to test it.
What I am looking for is a robust system, but also not terrible to manage for non “super-technical” person.
Also BTRFS was a little dissapointing after a few years using it, and even with its improvements over the years…I think that continues to be a little inestable to production scenarios (In my honest opinion but I can be wrong).
I did some quick tests with Truenas Scale, and the problem is that the only supported way to install apps on it is throught their app catalog TrueCharts… in which Urbackup isn´t.
It would be very nice to have Urbackup inside this system.
Of course you can (with some hacks) install Urbackup directly on their system (it is Debian based), but it´s an unsupported way by Truenas (they don´t allow direct package installs), and may have conflicts or problems with Truenas periodic updates.
I’ve been running a Ubuntu 20.04 server with ZFS for a Urbackup server for a while. only downfall with running ZFS is if you expand and your mount disconnects after reboot the service will start but backups will fail.
Another interesting difference is (and one of the reasons why I chose BTRFS in my setup), that cross-backup deduplication does only work on BTRFS, since ZFS has no reflink copy. So you will get deduplication between backups of the same source, but not between sources.
Since my setup has a lot of shared files between different hosts, this made a significant difference to me.
(Given that: I don’t have the amount of RAM to enable ZFS online deduplication, that would change the picture.)
I’m not sure if I get it right @capi - are you saying that if client A has a big file foo.bar and client B has the same file (either from the beginning or later at some point) then it is stored twice on the server?
ZFS. ALWAYS choose ZFS. I’m running both zfs and btrfs instances of Urbackup. And I never need to login to the ZFS instances. The btrfs instances are causing me lots of issues, lack of performance and cause disappointed customers. One server is doing 200MB/sec diskio for Btrfs, while backing up terribly slow. ZFS does not have reflink copy, because it’s snapshot method is great.
Always go for ZFS, really. The only thing holding me back from dropping btrfs is that infscape backup doesn’t support it…
I’m using Ubuntu 20.04+ZFS on many servers and have no issues at all. I’m also testing the waters with openSUSE+ZFS on one server and it’s the same, stable so far for about a year now.
In the past I was usunig FreeBSD+ZFS for UrBackup servers and it worked fine, but system management (not ZFS related) was not scaling in my case.
Do you think that ZFS compensates despite having the problem of reflinks. I mention it because of the waste of space it represents in the case of duplicating files between datasets.
I have been doing some tests with Truenas Scale because its GUI seems to be very good for managing the storage. The problem is that the integration with Urbackup is too manual for me. Docker integration is not well documented for someone without a lot of knowledge on the subject… and as I said before, installing packages directly on the system is not supported (although possible).
One possible option, speaking of Thunderas Scale, is to have a virtual machine running Linux with Urbackup inside Truenas Scale itself and connect the storage via iSCSI… but the performance is possibly very poor.
Next test I’ll do with Ubuntu+ZFS, even though it doesn’t have the nice GUI for storage management.
For BTRFS: If it’s the same file, it’ll be a refcopy, so only contain one copy on disk, even between clients.
For ZFS: It will be “one copy” if you use ZFS’ Online Deduplication feature, which requries a lot of RAM. It will be two separate copies, if not, since ZFS-on-Linux does not support refcopy between subvolumes, and every backup of every client is a subvolume. See COW cp (--reflink) support · Issue #405 · openzfs/zfs · GitHub
In my spare time (very little) I’ve played with UrBackup on different distros but never got the dedupe to work. I gave up after little effort and run UrBackup on Windows now for about 10 PC’s.
I do only image backups.
I’d like to get dedupe working but can it really work when you’re only doing image backups?
Ah you mean the copy-on-write backups scenario, you are right.
But that is one particular way of server configuration. There is also other choice where you don’t use that special Urbackup + zfs integration.
Nothing is stopping you from using ZFS pool as a traditional storage location. You still get all ZFS benefits and file-level deduplication between clients is provided by Urbackup. This is how I configure my servers for file backups and believe me, file is stored only once between multiple clients.
Is there anything further you need from us to consider the pull request? It would be really helpful/reassuring to get a ZFS-enabled Docker image straight from the source.
I keep rereading the docs and the blog but I’m not certain I understand how CoW image backups work on ZFS. Are you really using a special image format with delta shadow files, or are you creating/snapshotting a zvol and only writing blocks that differ during any given pass?
Many thanks for merging the Dockerfile changes with optional ZFS support!
And congratulations on releasing 2.5.25.
If you get a chance, it’d be awesome to see an official tagged 2.5.25_amd64_zfs container in Docker Hub. But either way: thank you. And if I can be of any help please let me know. Cheers!
For me it’s always ZFS, but if I had to go with btrfs then only on SUSE where it’s a first class filesystem.