Ive been using urbackup 1.3.2 for over a year now and I love it.
I noticed the backup server’s drive becoming very full.
Filesystem Size Used Avail Capacity Mounted on backup/urbackup 12T 12T 432G 97% /backup/urbackup
In the urbackup statistics it says its using about 43TB total.
Images 1.38 TB Files 42.36 TB All 43.74 TB
From what I have been able to guess urbackup does cleanup based on the global soft quota (30T) and its own statistics (44T) and not the actually filesystem space used (12T).
Global soft filesystem quota: 30T
My cleanup time is,
Cleanup time window: 1-7/3-12
And I do see some deletes in the ‘Activities’ on a daily basis, but its never enough to get below the soft quota and my storage space is going to fill up completely soon.
I attempted to run urbackup with “–app clean --cleanup_amount 3T” and it would just zip right through the cleanup processing saying it would remove 3T but it never does anything and exits almost immediately.
I thought maybe there is a bug in 1.3.2 so I upgraded the box to the latest 1.4.10. I then issued the same manual cleanup command and this time it actually did delete some backups and clear a little bit of space.
2015-11-16 09:37:02: Cleaning up 3 TB on backup storage 2015-11-16 09:37:02: Database cache size is 400 MB 2015-11-16 09:37:02: Starting cleanup... 2015-11-16 09:37:02: Freeing database connections... 2015-11-16 09:37:02: Deleting client with id "5" name "sql" 2015-11-16 09:37:02: Removing image with id "3638" 2015-11-16 09:37:12: Removing image with id "4259" ... 2015-11-16 10:18:29: Updating del files stats: 99% 2015-11-16 10:18:29: File processing speed: 307.692 files/s 2015-11-16 10:18:29: Updating del files stats: 100% 2015-11-16 10:18:29: Updating file stats... 2015-11-16 10:18:29: Done updating statistics. 2015-11-16 10:18:29: Cleanup successfull.
After it was finished it had freed up 1% of the actual filesystem to 96% full.
That test gave me hope and I tried to run the exact same command again but this time with 5T to cleanup, and now it does not delete anything again. It just races through the process and exits. I cant get it to clean up space.
cd /usr/local/var/usr/local/sbin/urbackup_srv
–plugin /usr/local/lib/liburbackupserver_urlplugin.so
–plugin /usr/local/lib/liburbackupserver_cryptoplugin.so
–plugin /usr/local/lib/liburbackupserver_downloadplugin.so
–plugin /usr/local/lib/liburbackupserver_fsimageplugin.so
–plugin /usr/local/lib/liburbackupserver_httpserver.so
–plugin /usr/local/lib/liburbackupserver.so
–http_root /usr/local/var/urbackup/www
–workingdir /usr/local/var
–user urbackup
–http_port 55414
–logfile /var/log/urbackup_srv.log
–no_daemon
–app cleanup
–cleanup_amount 3T
-v
-v
-v
2015-11-16 10:35:29: done. 2015-11-16 10:35:29: Loaded -url- plugin 2015-11-16 10:35:29: Loaded -cryptoplugin- plugin 2015-11-16 10:35:29: Loaded -download- plugin 2015-11-16 10:35:29: Loaded -fsimageplugin- plugin 2015-11-16 10:35:29: Starting HTTP-Server on port 55414 2015-11-16 10:35:29: HTTP: Server started up sucessfully! 2015-11-16 10:35:29: Shutting down all database instances... 2015-11-16 10:35:29: Opening urbackup server database... Backupfolder not set 2015-11-16 10:35:29: Backup destination cannot handle subvolumes and snapshots. Snapshots disabled. 2015-11-16 10:35:29: Transitioning urbackup server database to different journaling mode... 2015-11-16 10:35:29: Cleaning up 5 TB on backup storage 2015-11-16 10:35:29: Database cache size is 400 MB 2015-11-16 10:35:29: Starting cleanup... 2015-11-16 10:35:29: Freeing database connections... 2015-11-16 10:35:29: Space to free: 5 TB 2015-11-16 10:35:29: Deleting old logs... 2015-11-16 10:35:29: Done deleting old logs 2015-11-16 10:35:29: Cleaning history... 2015-11-16 10:35:29: Rewriting daily history... 2015-11-16 10:35:29: Rewriting monthly history... 2015-11-16 10:35:29: Rewriting yearly history... 2015-11-16 10:35:29: Done cleaning history 2015-11-16 10:35:29: Updating statistics... 2015-11-16 10:35:29: Updating image stats... 2015-11-16 10:35:29: Updating deleted files... 2015-11-16 10:35:29: Updating file stats... 2015-11-16 10:35:29: Done updating statistics. 2015-11-16 10:35:29: Cleanup successfull.
I believe I am asking it to free up 5TB of space, and not free space until you are currently using only 5TB. But either way it isn’t deleting.
The server is FreeBSD 9.3 and the storage is on ZFS with lz4 compression enabled with UrBackup 1.4.10 server.
backup/urbackup refcompressratio 1.52x -
backup/urbackup written 12.1T -
backup/urbackup logicalused 18.4T -
backup/urbackup logicalreferenced 18.4T -
