Hi everyone, I am trying to fix the report in the Backups section, because currently the statistics are wrong.
For example, we have a client with name “PC-1” which creates a full initial backup with 60GB.
Every 20 hours an incremental backup is made, which accordingly to urbackup, occupies 5GB, but thanks to the btrfs snapshots the space really occupied is about 200MB.
When the first full backup exit from retention period, and gets deleted, the server loses trace of the occupied space.
There are two problems:
Urbackup does not know how much space the incremental backups takes.
Urbackup loses trace of the occupied space when it deletes full backups.
We tried to fix these two problems by enabling the quota on the btrfs, and by using this btrfs-du command(https://github.com/nachoparker/btrfs-du) we extract the data of the exclusive snapshot space.
Even if these values are far from perfect, them are more reliable than the one calculated by urbackup.
Do you have any suggestion on how to detect where and how to fix the second problem? Ideally we should find the code lines where the cleanup is made, and identify the latest backup that will not be deleted.
We are using the btrfs file system with quota enabled.
I have seen this bug has been around for many years, is there an official solution to this or is it really impossibile to fix?
I am trying to follow a different approach, instead of calculating at every backup the correct size, I am trying to run a cronjob which updates every n minutes all the backup sizes.
I tried to edit a simple backup size_bytes inside the /media/Backup/urbackup/urbackup/backup_server.db but even if I updated the value, the web interface is still displaying 16KB of used space.
Does urbackup has a cache internally?
Edit: I was wrongly trying to edit the backed up database, updating var/urbackup/backup_server.db works!
Yeah, if a file is partially modified it (sometimes) adds the whole file to the used storage.
It doesn’t move the 60GB to the other, still existing, backups. But the total storage used by the client should still contain the 60GB…
I have to admit that I improved the statistics a bit w.r.t. image backups in the appliance. But not for file backups (yet). The technique there could be used for file backups as well. The problems
It’s very dependant on the file system being btrfs. Which is sub-optimal for UrBackup which should work with every fs
It assumes there isn’t a btrfs offline block-level deduplication tool (like bees) in action.
It kind of works like btrfs-du only it can incrementally update the information as new image backups come in and get deleted without enabling btrfs quotas (as that tanks btrfs performance…).