Hello, so i have a urbackup server that backups multiple computers windows and linux but only File backups no image. And i was tasked to backup the backuped data to a Nas storage device which will be offline all the time and only online when updating the backup once a month or two months or so .
My initial tought was to just copy the urbackup backup volume with rsync over ssh . I have done this succesfully . From 1,63 TB of data on urbackupsrv i got 6,98 TB on NAS and then the file copy stopped because it did not have enought space left on the NAS to continue . Urbackup and the Nas both are btrfs file systems.
The rsync command i used is :
“rsync -avP --delete urbackup@192.168.0.160:/mnt/backup_partition/$1 /volume1/backup_8_b”
where $1 represents a parameter to a bash function which is just the different folders in the backup_partition . Im starting multiple rsync processes concurrently to speed it up a bit. Each folder in the volume gets a own rsync process .
So my first problem is the size , i was expecting for it to be sub 7TB as the nas has 7Tb of free space.
Kudos to urbackup space management. I guess this is probably because im copying symlinks of incremental file backups as files or something to that end ?Help with understanding would be appreciated .
My first question thus is what would be the best solution to copy everyrelevant data from the backups but space efficient with intact symlinks ? Is the -H for rsynch command as explained here the solution for this ?explainshell.com - rsync -a -x -u -v ?
My second question is what else do i need to copy from the server to be able to restore the backup in case of fatal unrecovarble failure ? Do i also need the urbackup user folder ? Other settings files folders ?
My third question is what would be the best way to do a recovery from the Nas . Setting up a blank new server and just copy over all the files? or do i need to do a snapshot of the current urbackup ? The urbackup is a virtual maschine but i can only snapshot the linux system volume because of space requirements and i can do that only if i detach the volumes where the backups are happening because of functional limitations of Xenserver which is in use as the underlying virtualization enviroment .
My fourth question is data integrity while being copied , do i need to turn the urbackupsrv.service off ? i have done that currently with my first test . If this is the case , is there a command line command or other solution where i can look up if there are any ongoing backup activities and wait for the to finish ? Is it even necessary to wait for it to finish ?
Any ideas or remarks or better approaches are highly appreciated.
LG Marek Stepan
P.S. Currently after first couple weeks of using urbackup im absolutely loving it ! Not only the software but the fast and good support i got on the forum here. I have had a couple bad experiences with f.e. openvpn forum where i was so disgusted that i will never again use openvpn. I will definetly want to persuade my chef to purchase licenses for the client computers to speed it up.
My solution would be to image the backup storage ideally using an lvm, dattobd or other snapshot so it’s consistent (in my case Windows VSS), the nightly database backup can be restored from the restored data drive to a new blank server install if such an image is needed.
Hello ,firstly thank you for your input .
I have never worked with such snapshot mechanism outside of the integrated solutions in virtual enviroments .
Do you have any experience with dattobd on linux or other solutions ? If so, is it possible to save the snapshot directly to a NAS or some FTP folder ? What are the disk space requirements for such a snapshot ? if f.e. my vm is 50gb OS partition and 12 TB urbackup partition , do i need at least 12TB&50GB size free on the snapshot location ? because i only have 7ish TB available.
No experience to speak of, especially not with dattobd, sorry, & now I’m on Debian Bullseye for my Linux boxes I’m not going to gain any, it’s apparently broken with the kernel version Bullseye uses.
LVM snapshots can be scripted though, you need enough free space to create the snapshot, which needs to be big enough for all the data that might get written to the main volume while the backup happens. I couldn’t tell you how much is actually required, but people who use LVM snapshotting typically leave around 10% of the pv free when they set up their logical volume(s).
Producing an image off that is simply a case of mounting the snapshot read only as soon as it’s created, & just dd the entire thing to a big raw image file on your remote storage by all means pipe that through gzip if you’d prefer to save space on the remote store it shouldn’t complicate a restore that much, then unmount & remove the snapshot when done. SMB or NFS will do to get it across the network.
Restore would just be a case of re-create the new empty backup storage space and dd your image straight back to it. Or you could look at things like tar, or fsarchiver to create a backup from the snapshot, how well those handle all the symlinks & hard-links UrBackup uses I couldn’t tell you though, dd won’t care, does the thing block by block & puts it back the same way, doesn’t even need to be a supported filesystem to work.
Just don’t ask me for specifics for any of this my UrBackup setup is 100% Windows.
thank you for your response , i research about lvm snapshost yesterday after your post , and as i have spannned my lv on the whole vg i need to shrink them , im probably gonna try that or im still thinking about , just copying some of the backups from urbackup , as the way i tested it , it seems to me that no matter what i copy from the urbackup backup volume (incremental or full backup) on the file system if i rsync copy it it is always the whole file ,i will consult with my boss if like 2 or 3 backups are enought then the space requirement would be sufficient .
Since you are using btrfs already, did you consider taking advantage of btrfs snapshots for that purpose?
I’m doing something similar with ZFS backed servers. UrBackup storage directory and /var/urbackup are both on a separate ZFS datasets (btrfs subvolumes in your case). I’m using snapshot+replication to do a periodic cold storage backup of my backups
I will look into that tool . And also thank you for the information that it is necessary to bakup /var/urbackup .
Does that mean that with a new install of urbackup server and copying over the /var/urbackup folder i basically restored the urbackup configuration and the references to the backups on the backup volume ? Or do you perhaps know if i need to look into some urbackup folder in /etc also ?
Your /whatever/storage and /var/urbackup are absolutely required. I did it in the past - moved these two to a new server, made sure UID/GID mappings are the same on both servers and UrBackup continued working just fine.
In addition you may want to include:
/var/logs/urbackup logfiles if you want
/etc/urbackup/ where backup location and various scripts go
/etc/defaults/urbackupsrv or /etc/sysconfig/urbackup-server depending on distribution you are using. You don’t need it if you didn’t change the default settings.
If you are using Apache/Nginx instead of built-in web server then appropriate config files and SSL cert+key
