I have just recently followed the instructions to migrate the backup store from a local Windows server Running on the client machine, to a new Linux nas server.
The data was migrated by mounting the iscsi Target (NTFS) to the new servers, and running rsync -HvP to the nas-pool (ZFS)
The rsync process was showing that it was successfully creating hard links in the new file system As it dredged through the transfer process. After migrating the data, I then followed the final steps of transferring the database files over to the new server. All was put in place and chown -Rh Was applied to update the owner and group to urbackup:urbackup On both the database files and the backup data… I then ran remove-unknow…
It was a bloodbath of the following.
2024-08-05 15:15:17: WARNING: Refcount of "/mnt/nas-pool/UrBackup/(redacted)/.directory_pool/zR/zrleTLpz9J1712332639203808671" is zero. Deleting pool folder.
2024-08-05 15:15:17: WARNING: Refcount of "/mnt/nas-pool/UrBackup/(redacted)/.directory_pool/zR/zrw23wQDgO1715250718432799281" is zero. Deleting pool folder.
2024-08-05 15:15:17: WARNING: Refcount of "/mnt/nas-pool/UrBackup/(redacted)/.directory_pool/zU/ZU6NgVyy5V17183501173532163765" is zero. Deleting pool folder.
2024-08-05 15:15:17: WARNING: Refcount of "/mnt/nas-pool/UrBackup/(redacted)/.directory_pool/zU/ZU9k5c5Xs217179167923098847578" is zero. Deleting pool folder.
Luckily, The data is safe, as I wisely mounted the iscsi as read-only, and the target and old server data is still alive and well. However, this is making me quite perplexed on what I did wrong as I followed the migration procedures as described in the admin manual, migrated it in a way that preserves hard links, Change the ownership to urbackup so the server can rwx The data and its database, and made sure the new updated path was changed in the server…
I let the process finish as the data’s not actually lost and figured I’d just let it run through to see how far the extent of the damage would be. out of 2.1TB of Backup data, It is whittled down to a measly 375GB…
This suggests that it is recognising some of the data belongs and shouldn’t be removed, but the rest got Thanos snapped out of existence for some reason.
Some Clues and Important info:
-
The server is running on Proxmox, with Proximox handling the ZFS volumes, With a single subvol handed over to the guest to hold the data.
-
Debian is running in an unprivileged container, However, the correct permissions have been set (
chown 100000:100000) on the host, to pass the iscsi target mount point over to the containers root, so rsync could get full acssess during the migration. I don’t think this would be much of an issue, As far as i can tell Urbackup doesn’t run as root and makes its own user and group. The new destination path and the database files all have the correct permissions set. -
Going through The backup directory I noticed that the .directory_pool dir Has been completely gutted with nothing left. it was there before.
-
The rest of the directories seem to be intact, with data inside, However, some simlinks looked like they were copied over. They are pointing to the Iscsi target Mount Point on the host, But strangely not the iscsi mount point of the container?
If anybody has any idea what’s going on here, I would greatly appreciate it. I’ve been doing three days of my own attempts, Transferring the data over again three times Just in case I’ve got something wrong, but yet the result is always the same. Rsync is showing that it’s copying the data over and making the links, yet the remove-unknown procedure Just wants to gobble away all the backup data thinking the reference count is zero…
I already attempted using the built in UrBackup migrate_storage_to file method, while mounting the Directory via Samba, yet the documentation I’m looking at is completely mixed with some saying it can handle hard links with others saying it can’t… I just get an error message in the UrBackup server logs Stating X is not a re-parse point.
The only final hail Mary I can think of is getting a hard drive of significant size to hold the entire backup in one go, To use the built in migrate feature. Then on the other end, set that as the backup destination, And run the migrate Feature again to the location I want it on the nas… I don’t think I have a external hard drive with the capacity of doing this however, and don’t want to spend money, But if that’s the only solution that people can think of, then so be it.
I’m sure one question will be why don’t I just mount the iscsi volume directly and work off of that… Well, the paths are already going to be borked, as Linux does not follow the N:/ Path paradigm, So it won’t be a direct plug-n-play anyway with some form of migration needing to be done, And I really don’t want to run remove unknown on the base data given what I’m experiencing.