I’ve been using (and learning) urbackup for several months now as a docker container with the storage being a 4TB disk in the NAS. It works great, albeit I’ve had my fair share of “how does this work” moments.
Although not an issue at the moment, I do need better understanding of the phenomenon I am observing. This has to do with File Backups - Full and Incremental. I finally got my head wrapped around incremental backups basically contain links to every file that has been backed up since the last full File Backup. In my case I am specifically including a specific folder containing my photos. The du size of those photos is nearly 85GB. The initial full backup of those files has been deleted. I am retaining incrementals on a schedule. No more full backups (file) are scheduled (disabled). Incremental backups (file) capture all newly added photos. All works as expected. On the Backups listing tab, I can click on the Photos folder and navigate down to any particular file all the way back to my original Full File Backup. I can even download said file.
What I am seeing reported in urbackup Statistics tab for the size of those file backups, it is reported as 5.48GB. The du linux command on the storage folder in the NAS shows 82GB which is close to what I would expect the 85GB original du size on the original folder (not the backups).
What causes this discrepancy?
edit:: Also, the sum total of all backups (File, Image, Full, Incremental, All Machines) as reported in urbackup Statistics tab is 385GB. The du size on disk of those backups is 512GB. As I stated to start, I am using a 4TB drive as storage, so this isn’t a huge issue…yet. It does concern me however. The graph on the statistics tab does show increases and decreases as archived backups fall off. But this is a rather large discrepancy.
BUT, since you mention removing the initial full file backup, I suspect you are on btrfs for the backup storage?
The sizes shown in the “backup history” is the actual data added to the backup storage. So if the inital full was lets say 100GB, and you make an incremental, the new incremental will only show the added data. If you then remove the full the incremental still only shows the added data amount each time you run a new incremental.
IF you are on btrfs, “normal” du is not the one to use, use: sudo btrfs fi us /path/to/backupmount
fi = short for filesystem
us = short for usage
If you are NOT using btrfs, I THINK each saved incremental, in files that are changed, they remove the symlink and uploads the whole file, I’m not sure other filesystems on linux will be able to handle the “paching” mechanic so each changed file is uploaded to its full size.
So if you change a single byte on lets say a 5gb iso file, the whole file will be upladed again, I THINK! Someone please correct me if I’m wrong.
@anon16079863 My NAS uses XFS as the file system. I do not use btrfs. The command I use on the backup disk is du -h -s /path/to/backupmount (-h is human readable so output is in GB, MB, KB, not just a string of several numbers; -s is output for all subdirectories).
As I mentioned, this isn’t a problem…yet. I have plenty of empty space on the 4TB disk devoted to the backups. My concern is, do I have to perform the linux du command on the disk to properly ascertain how much disk space is used, rather than the Statistics tab in the web console? I don’t mind doing so. I’m in the console several times a day. But if so, then I know I cannot rely on the space utilization in the urbackup web console to let me know when I may need to add another disk to the mix. Not a problem now … But I’ve only been running the server for about 3 months.
Thank you very much. I have always been aware and somewhat confused on df vs du, but the links provided help me a bit more. It is a continuous journey of learning! For the near future I’ll default to using the du command in console, as that appears to be what my system will balk at, not the reported number in urbackup Statistics tab.
Again, thank you for the links and your help. Greatly appreciated.