Remove-unknown no longer completes

Hi guys. I have run a small urbackup server for about 2 years to do a file backup of two PCs (total backup server size is about 3 TB). I have run the remove-unknown function once a month to clean up the database/files and have never had a problem with it (it generally takes 2 days to complete). A number of months ago there was a urbackup update and the remove-unknown function no longer completes. At present, it has been running for 3 months! It’s doing something; it’s always got a substantial amount of either CPU or IO utilization. The log shows only the following types of messages:

2018-07-09 06:54:51: WARNING: Adding missing directory reference of pool nRbtDavlMd1497462779147202437 to “/var/urbackup_files/THUNDERSTORM/170612-1927/.hashes/C/Windows/Globalization/ELS”
2018-07-09 06:55:03: WARNING: Adding missing directory reference of pool nRbtDavlMd1497462779147202437 to “/var/urbackup_files/THUNDERSTORM/170613-0031/.hashes/C/Windows/Globalization/ELS”
2018-07-09 06:55:10: WARNING: Adding missing directory reference of pool nRbtDavlMd1497462779147202437 to “/var/urbackup_files/THUNDERSTORM/170613-0958/.hashes/C/Windows/Globalization/ELS”
2018-07-09 06:55:17: WARNING: Adding missing directory reference of pool nRbtDavlMd1497462779147202437 to “/var/urbackup_files/THUNDERSTORM/170613-1505/.hashes/C/Windows/Globalization/ELS”
2018-07-09 06:55:24: WARNING: Adding missing directory reference of pool nRbtDavlMd1497462779147202437 to “/var/urbackup_files/THUNDERSTORM/170613-2230/.hashes/C/Windows/Globalization/ELS”
2018-07-09 06:55:33: WARNING: Adding missing directory reference of pool nRbtDavlMd1497462779147202437 to “/var/urbackup_files/THUNDERSTORM/170614-0332/.hashes/C/Windows/Globalization/ELS”

Should I just delete all the file backups and start over? Any help in resolving this problem would be appreciated!

Thanks,
James

You do not need to run remove-unknown regularily, or at all usually. I wonder where you got that notion?

I’d recommend starting a full file backup for each client, and then if you must, run remove-unknown after the problematic backups have faded mostly away.

Will do, thank you!

A long, long time after…

I have notice the size of all of files exced the amount display in Web interface
After some search, I have find a large cache folder for each client : .directory_pool which are very heavy.
The problem is some of file inside this folder ar no longer used.

My solution : looking for every file who are not use in current backup (not hardlinked)

sudo find /your_backup_directory/*/.directory_pool -type f -links 1 -print -exec rm {} ;

Explanations:
-type f = file
-links 1 = number of links, if only 1 link it not used as cache
-print = display result in console

  • exec = execute commande remove file who match
    {} = name of each file found by find before exec
    ; = syntax of end of action

/*/ = use to enumerate each client folder inside your backup directory
.directory_pool use a point before, this is the reason why it was hide