Deduplication without BTRFS/ZFS

Firstly, I want to thank Uroni and anyone else involved with UrBackup development. I’ve been a UrBackup user for about 3 years now. UrBackup is a wonderful piece of software and replacing it has been really difficult - I’ve had to cobble something together and it’ll be a while before it works as well as UrBackup.

But - I feel it’s a necessity because I don’t feel I can trust BTRFS again. As soon as I try to do anything major with it, like add another disk, heavy I/O or upgrade the kernel - it has far too high a risk of breaking. I’ve already reinstalled the system a couple of times due to BTRFS issues and now a third time is just too much.

I was using BTRFS for deduplication, since UrBackup doesn’t support deduplication in software. I have previously tried using EXT4 and symlinks but it was just too slow to take regular backups. My understanding is that right now there is no deduplication in the software itself - it’s basically done at the filesystem level with snapshots in BTRFS or filesystem deduplication in ZFS which uses lots of memory?

Right now, I’ve switched over to Borg Backup. This does deduplication in software, so I can still use EXT4. But, it doesn’t have centralised management, a web interface, statistics, etc, so I’m having to build a lot of that myself.

I will keep an eye on UrBackup and if deduplication is ever supported properly in software, then I will jump back straight away.