I am deploying UrBackup in TrueNAS with ZFS and i have some questions about how to properly set my datasets.
Can you advise if i should set the following dataset properties in TrueNAS or if UrBackup will set them automagically?:
** Deduplication=on/off
** Checsum=sha256 (images) / sha512 (files) [admin manual Section 6.3.1 & Section 6.3.2]
** Record Size = 512K (Images) 128K (files) [admin manual Section 6.3.2]
Should we enable deduplication on the dataset where the “backup folder” resides?
It appears, based on the example in the admin manual section 11.7.1, that i should enable deduplication on this dataset but it is unclear to me because to enable ZFS compatibility we must create a dedicated dataset for the image backups and another dataset for the files backups. So, it appears that the “backup folder” would be unused. is this correct?
Should i use “RAW” transfer mode for the image backups in a ZFS file system?
The admin manual section 8.5.3 implies that in a BTRFS file system the RAW image file format is used. However it makes no mention to the recommended transfer mode in a ZFS file system. I assume it would be “RAW” as well but i just want to confirm.
Should i use “RAW” transfer mode for file backups?
The admin manual only makes reference to image files in BTRFS file systems but makes no mention to files backups. Similar to the question above, i assume I should use “RAW” transfer mode for file backups as well but i wanted to confirm.
help?
Edit: Added links to the referenced sections of the administration manual.
Message received. This is an open source project with a very dormant community and very minimal support from developers. At some point I even questioned if the project was dead but some slight activity in the forum proved that was not the case.
Regardless of the lack of support, I will do the right thing and post my findings in the hopes it can help somebody else down the line.
It seems that UrBackup does not set the dataset properties automatically. However:
** Deduplication should be turned off for the dataset in TrueNAS. I’m not an expert at this but my understanding is that TrueNAS performs block level deduplication which is not recommended in most cases as it is very resource intensive (see Can i use a partition for a SLOG? - TrueNAS General - TrueNAS Community Forums for more information). It appears that UrBackup performs “file level” deduplication which is much more efficient than block level deduplication which it does regardless of if deduplication is turned on/off on the dataset.
** Checksum-sha256 (images) / sha512(files) I left these settings like this. I don’t know enough to advise different settings.
** Record Size = 512K (images) / 128l (files) I left these settings like this. I don’t know enough to advise different settings.
Deduplication on the dataset where the “backup folder” resides should be turned off. (see response of item #1). To answer my own question, this folder will not be empty as UrBackup will create links in here pointing to the image and file datasets after their creation.
Yes, use the “RAW” mode for image backups in ZFS file systems.
There is no “RAW” transfer mode for full file backups but “Hashed” should be used. I assume UrBackup uses it for their “file level” deduplication. Similarly, “block differences - hashed” should be used for incremental file backups to allow UrBackup to transmit only the blocks of data that changed, instead of the entire file. This is very useful for big files that changes regularly such as a VeraCrypt volume.
I think that is it, this item is considered resolved by the OP
I have set my system up very similar to yours. One thing that I believe that I do differently is that I only do file backups, no images.
After about a year, my urBackup server says there is 735GB of backed up client data, and looking at the Datastore from TrueNAS, it is taking 1.1 TB of ZFS storage, with a compression of 2.2. De-duplication is off
@smcclos , it is interesting to see your storage space use.
How much data are you backing up?
I mean, are you backing up 100GB of data and URbackup is saying that the backups are taking up to 735GB?
if that is the case, have you enabled ZFS on urbackup?
I ask because, on my system, the incremental backups doesn’t seem to be taking any space. Which makes sense because i haven’t change these files in a while. But also because i think Urbackup is using the snapshot feature of ZFS to create the incremental backups which basically doesn’t take any space.
TBH, I have not calculated how much I am backing up. I just add a machine to backup, and watch my consumed space on TrueNAS, and make sure I don’t fill up my pool.
I would have to day that for the 735GB of data I have in backups, that the data I am protecting is around that same number.
As for if ZFS is enabled in my backup; that is a resounding yes. Based upon how I set up my server, it is a BSD Jail on the TrueNAS server and the backup repository is in a dataset in my pool. I used this document as a basis to build my server:
Code:
Create the urbackup jail:
iocage create -n urbackup ip4_addr="vnet0|10.10.0.39/24" defaultrouter="10.10.0.1" vnet="on" boot="on" -r 13.1-RELEASE -b
Go to the jail console:
iocage console (name of jail)
Install needed packages: (respond "y" if prompted)
pkg install -y nano curl wget
download Urbackup Source into the jail:
wget https://hndl.urbackup.org/Server/2.5.27/urbackup-server-2.5.27.tar.gz
Extract the archive:
tar xf urbackup-server-2.5.27.tar.gz
Navigate into the newly created folder:
cd *.27
Then :
./configure --enable-embedded-cryptopp
Then :
make -j4
Then:
make install
To test do:
urbackupsrv run -u root
Browse to:
"jail IP":55414
Back in the jail stop urbackupsrv:
ctrl-c
To make urbackup run at jail boot :
nano /etc/rc.local
and add to the new file :
#!/bin/sh
/usr/local/bin/urbackupsrv run -d -g 104857600 -u root
To end and save :
crtl-x
y
enter
To make it executable :
chmod +x /etc/rc.local
"exit" to leave the jail
"iocage restart (jail name)" to restart the jail then check you can browse to port 55414 as before
1.SSH into truenas
2.Get into the Urbackup jail. “iocage urbackup” most likely
3. Issue the following commands to setup ZFS for the “image backups” (Adjust per your enviroment)
@smcclos I’m not entirely sure you are correct.
Like you said, In a regular installation the “backup directory” in the jail is linked to a dataset in the TrueNAS host that uses ZFS.
Based on this, yes, ZFS is inherently implemented but only partially. With this method only TrueNAS would be capable of creating snapshots.
Why is this important?; because UrBackup would be able to create/delete child datasets and snapshots for each backup which is an ideal use case considering the advantages in performance and space savings of ZFS snapshots.
