ZFS Dataset deduplication in TrueNAS CORE

I am deploying UrBackup in TrueNAS with ZFS and i have some questions about how to properly set my datasets.

  1. Can you advise if i should set the following dataset properties in TrueNAS or if UrBackup will set them automagically?:
    ** Deduplication=on/off
    ** Checsum=sha256 (images) / sha512 (files) [admin manual Section 6.3.1 & Section 6.3.2]
    ** Record Size = 512K (Images) 128K (files) [admin manual Section 6.3.2]

  2. Should we enable deduplication on the dataset where the “backup folder” resides?
    It appears, based on the example in the admin manual section 11.7.1, that i should enable deduplication on this dataset but it is unclear to me because to enable ZFS compatibility we must create a dedicated dataset for the image backups and another dataset for the files backups. So, it appears that the “backup folder” would be unused. is this correct?

  3. Should i use “RAW” transfer mode for the image backups in a ZFS file system?
    The admin manual section 8.5.3 implies that in a BTRFS file system the RAW image file format is used. However it makes no mention to the recommended transfer mode in a ZFS file system. I assume it would be “RAW” as well but i just want to confirm.

  4. Should i use “RAW” transfer mode for file backups?
    The admin manual only makes reference to image files in BTRFS file systems but makes no mention to files backups. Similar to the question above, i assume I should use “RAW” transfer mode for file backups as well but i wanted to confirm.

help?

Edit: Added links to the referenced sections of the administration manual.

anyone?

Message received. This is an open source project with a very dormant community and very minimal support from developers. At some point I even questioned if the project was dead but some slight activity in the forum proved that was not the case.

Regardless of the lack of support, I will do the right thing and post my findings in the hopes it can help somebody else down the line.

  1. It seems that UrBackup does not set the dataset properties automatically. However:

** Deduplication should be turned off for the dataset in TrueNAS. I’m not an expert at this but my understanding is that TrueNAS performs block level deduplication which is not recommended in most cases as it is very resource intensive (see Can i use a partition for a SLOG? - TrueNAS General - TrueNAS Community Forums for more information). It appears that UrBackup performs “file level” deduplication which is much more efficient than block level deduplication which it does regardless of if deduplication is turned on/off on the dataset.

** Checksum-sha256 (images) / sha512(files) I left these settings like this. I don’t know enough to advise different settings.

** Record Size = 512K (images) / 128l (files) I left these settings like this. I don’t know enough to advise different settings.

  1. Deduplication on the dataset where the “backup folder” resides should be turned off. (see response of item #1). To answer my own question, this folder will not be empty as UrBackup will create links in here pointing to the image and file datasets after their creation.

  2. Yes, use the “RAW” mode for image backups in ZFS file systems.

  3. There is no “RAW” transfer mode for full file backups but “Hashed” should be used. I assume UrBackup uses it for their “file level” deduplication. Similarly, “block differences - hashed” should be used for incremental file backups to allow UrBackup to transmit only the blocks of data that changed, instead of the entire file. This is very useful for big files that changes regularly such as a VeraCrypt volume.

I think that is it, this item is considered resolved by the OP

I have set my system up very similar to yours. One thing that I believe that I do differently is that I only do file backups, no images.

After about a year, my urBackup server says there is 735GB of backed up client data, and looking at the Datastore from TrueNAS, it is taking 1.1 TB of ZFS storage, with a compression of 2.2. De-duplication is off

@smcclos , it is interesting to see your storage space use.
How much data are you backing up?
I mean, are you backing up 100GB of data and URbackup is saying that the backups are taking up to 735GB?

if that is the case, have you enabled ZFS on urbackup?
I ask because, on my system, the incremental backups doesn’t seem to be taking any space. Which makes sense because i haven’t change these files in a while. But also because i think Urbackup is using the snapshot feature of ZFS to create the incremental backups which basically doesn’t take any space.