Is there are best practices document or recommended procedure for migrating an existing UrBackup Server to a new hardware platform? I’ve read through the documentation but wasn’t able to get a sense of exactly how to perform the task. Any thoughts or past experiences?
2 Likes
I will be watching this thread closely and i also would like to get a sense of the correct approach to this problem.
1 Like
Hello edward.reed:
Are you moving the original backup storage hard drive to a new platform?
Or, are you wanting to move the backup data from the old hard drive to a new hard drive on a new platform?
All for now
We have yet to move a server successfully, it always works out faster to just install a new server then add the clients to that server and start backup from zero, the reason… well the data store… we are trying to copy the store from one disk to another, we have about 21’000’000 files, so far it looks like 30 days to do this, might as well re-back everything up as that store is only February (under 20 days) of data… so thats the first issue.
Next is the unique ID of the server for which it has a copy and so does the client, so in theory you can install the new server then take the server ID file and copy that back, if you have access to the server backend engine (we dont as its run on a QNAP NAS). I have requested a “backup” and “restore” feature in the server GUI as most of this would be fixed with that feature, it would also be nice to have a “move data store” feature in the GUI as there are so many links in the database that have to work.
The below link is interesting and answers a question i had, the new storage path has to be identical if you move the server, as it appears to hard code the links in the database, can be fixed by running a script, sounds like a GUI button is required for that if its a script.
Heres the word from the developer about it
This would be all new hardware, replacing the server and drives. Even worse, this would be changing platforms too. The existing UrBackup is running on a Windows Server with the backup data stored on a storage device connected over iSCSI. On the new system UrBackup will run in a Docker container and use local storage.
If I setup the new installation of the Server and move the server id file to the new installation, will the clients automatically register on the new server without user intervention? We have over 500 clients and about 50TB of backup data. Most client users don’t even know that UrBackup is in place. Giving them instructions to connect the client software to a new server is a recipe for disaster with no guarantee that they’ll actually perform the instructions.
Ive not tested this, but my understanding is yes, as thats the ID the client will present to the the new server. The new server will us UDP to find the client, the client will report in with the server ID it has (your old server) so as long as the new server has the old (restored) ID, the client will be seen and ok with the server to then inherit all new the new backup settings
ok. I’m still several weeks away from receiving the new hardware but I’ll give this a shot once it arrives. Thanks for the info.
PLEASE keep us updated on your progress. I am also looking to migrate an installation and it would be nice to refine and adopt some sort of coherent procedure sheet – even if it’s rough and draft-like. Hope all works well for you.
And THANK YOU for asking the question!
Any update on this?
Moving from unraid to truenas…
Can’t get it working
This is what is suggested.
urbackup also has a feature to move the backup to another drive without taking the backup system offline for non BTRFS file systems. You have to create a file named “migrate_storage_to” that contains the directory to which you wish for the backup to be migrated to. Urbackup will then continue backing up to the old location while doing the migration In the background.
https://www.urbackup.org/administration_manual.html#x1-9500010.8
This process is, however, quite time consuming. When I attempted this, it took fourteen days to complete on 800GB of backup data.
One of the biggest problems is windows to Linux… I’m sure there’s many people who started using urbackup on windows, Then got themselves a server or NAS and want to have the urbackup server running on that. (If this is what you are looking to do, and seeking answers on how to accomplish this, I will just say as a short answer… Don’t even try.)
The problem Is your backup relies heavily on the file system for its versioning and space saving techniques, Making heavy use of simlinks and hardlinks, As well as special file system features should they be supported, like snapshots on Btrfs, Or zvols for Image backups/FS Block level Deduplication on ZFS.
This heavy reliance on the file system is a double edged sword. On one hand, it means they don’t need to develop a bespoke database and data store system to handle the backups, Like you’d find with Duplicati. You can also trust that the file system will definitely be handling the data correctly. It also means that you can always plug the drive into any other system without the backup software installed, and just copy the data off. However, that means migrating the backup can be an absolute bore If the new target is not set up with the same storage architecture as the previous server.
This is why urbackup server itself has to migrate the data, as it needs to take the old structure and transplant it into the new location, asking for hard links and placing new simlinks in the new file system as required.
If on windows, It would be very likely that your backup drive was NTFS, and paths for the data would be something like “C:.…” This mean simlinks And server database entries will be pointing to locations following the Windows NT Directory Forest architecture. (This is why you have C,D,E,F… drives in Windows) When you tried to read these simlinks or transplant the server database in Linux, It makes absolutely no sense, because Unix operating systems take a single root to all paths approach.
Linux has no idea how to handle, C:,D:,E:,F:.… symlinks/paths because It comes from a architecture that treats each of these drives as their own root (Forest), and Linux just expects a single root, /… , where everything branches out from. (Single tree).
The closest I have gotten to getting a back up from a Windows server somewhat loading on a Linux server Was using rsync archive mode to preserve simlinks and hard links, But then also running a script over all the sim links to convert them all to relative paths. But this even still ultimately failed, As when running remove unknown, as suggested in the manual when migrating to fix simlinks, it just completely eviscerated the data.
This is because I think the database from the windows server contains paths like C:,D:\ ect, Which would not be present on a Linux system, so the server on the Linux side thinks that the file has nothing pointing to it and thus. “Removes Unknown”…
I think what urbackup desperately needs is a “migrate_server” command. This is where one urbackup server connects to another, and rebuilds the hole server on the target. The target server gets the data from the source server and rebuilds its database and data store completely. When done, The old server automatically shuts down and the new server takes over.
Or, another option. migrate_storage_to should migrate the data in a platform agnostic manner. Instead of system symlinks it should use mfsymlinks, and it should placed inode’s (Hardlinks) in a database for the other server to read. Then, when the other server Is linked to this as its backup directory, It recognises it is a migrated back up, and begins a maintenance task to rebuild the backup destination.