How many Maximum and Minimum Incremental Backups are required to capture a full image?

general discussion
1

In the manual, file backups have the following defaults:
Full file backup - 30 days
Incremental file backups - 5hr
Maximum incremental file backups - 100
Minimum incremental file backups - 40

My understanding is that in order to be able to reassemble a file backup set, you would need to have every incremental backup since the last full backup, else you would not be capturing changes to files since the last full file backup.

If you went with the defaults, you would potentially have only 40-100 incremental backups however at the end of the 30 days, 144 lots of 5 hours would have past, meaning that URBackup has deleted at least 44 incremental backups.

Would this not mean that if you did a full file restore on day 30, you would be missing about a third of file changes since the last full file backup?

If this is the case, shouldn’t URBackup automatically manage the min/max backups in a way that ensures that all changes are captured at the interval that you have set?

For example, if I set the following:
Full File Backup - 30 days
Incremental file backup - 1 hour
URBackup should keep a single full file backup and 720 incremental file backups which could be reassembled to restore all files as they where only 1 hour ago.

Some clarification and corrections would really be appreciated! I am having trouble with adequate disk space on the backup server and may be simply retaining more backups than required to adequately restore.

2

I am with the same doubts. I was going to put a similar message to this one.

I see this management of copies complicated and the manual does not clarify much. See if you can help us.

3

For what i understood, the difference between full and incremential is just that incremential does not copy files/block of files if they are not changed since previous backup. First incremental, it takes everything. Second, only the changes, …
So in fact, whatever you configure, you will always be able to retrieve all yours files.

I think the full file backup is useless.

4

@Danie_lPipe
I had been struggling with this concept too and also thinking that UrBackup should automatically manage the min/max backups rather than allowing you to set them.

However I guess we are making our view of the restore process more complicated than necessary, but I’m a beginner at this so please correct below if you think I am wrong.

So the first backup (#1), even if we chose it to be incremental, is actually forced to be a full backup, as @Montikore says.

Thereafter, incremental backup #2 just looks at the changes from #1 and copies those changes to the server.
Incremental backup #3 also looks at the changes from #1 and copies those changes to the server.

To do a restore, UrBackup simply bolts backup #1 (full) on to backup #2, or backup #1 (full) on to backup #3, etc, if we chose that particular time/date.

As for deletions (using the defaults), when we get to backup #102, UrBackup deletes backup #2 because we have set a max of 100 incrementals. So the only records we lose here are the history of file-deletes and file-overwrites done within 5 hours of the first full backup (#1). Likewise when backup #40 is deleted, we lose history from #2 to #40, and when #44 is deleted it is already time for another full backup.

In my view the most common thing that users might want to backtrack for is if they delete a whole project and only discovered that a month later, so if we use full backups at all, this figure (30 days or whatever) will determine far they can go back for retrieving deletes and overwritten files. But the additions are still there in the latest incremental backup.

So in the example @Danie_lPipe gives, it is true that some time on day 21 (100 backups * 5 hours / 24 hours), the first incremental backup #2 will be deleted. We lose the ability to restore only files deleted or overwritten up to 9 days before, but additions and subsequent overwrites still appear in day 22 onwards. In other words by the time the next full backup arrives we are keeping a trailing history of 21 days (30 – 9), assuming that both server and client PC are continuously switched on.

I think this is why @Uroni often refers to using incremental backups indefinitely, which would give an uninterrupted tracking history if that’s what you really want. In practice though I find most the most important backups for users are “just now”, “yesterday, and “last week”.

Sorry if that all seems rather long-winded but hopefully it’s the right concept.

Considering all that flexibility, UrBackup is still amazingly easy to use - great piece of work!

5

Don t overthink it.

For files, unless there s a bug, you can restore any files, even if you deleted an intermediate backup. Its hardlinked, so there s no management of number of backup to keep or whatever.

For block level backup (images) i wonder how it works exactly with cbt. But my guess is also test and report if there’s any issue when deleting an intermediate backup.

The concept of fulls isn’t only cosmetic, but really you need a very low frequency of full backup. Basically it s for fully rechecking that the files are good, redoing checksum, it also change transfet mode, in rare cases some bug need a full to be fixed.

The backup frequency/number of backup/archive settings allow to set how far and with which granularity you can go back. You can keep more recent backup, few old backup. Keep a fix ed number of backup or allow the disk to fill up untill it needs to delete backup automatically because there’s no more room to make space for the next backup

Before you ask. Changing the transfert mode is like a cpu<-> bandwith throttle, you trade mode cpu usage for less bandwith.

1 Like
6

Thanks for the replies so far.

There are two issues I’m trying to solve that I don’t think have been answered here…

SETUP

  1. SETS - File Backup
    -Ideally i’d like to keep an hourly incremental file backup of the file server for the past 7 days (180 increments).
    -I’d like a full file backup monthly to ensure integrity of the backup set (1 full file set)
    -I don’t need to keep more than 1 file backup set

  2. SETS - Image Backup
    -I need to image the server monthly
    -I need to keep only 1 image)

  3. OFFSITE STORAGE - Copying sets to external media
    -I need to copy the latest image and file backup of the server to external media to be carried offsite daily
    -I’ve written a basic script to copy the entire “server1” backup set directory to an external drive as well as the UrBackup database (as this is also on premise thus if the file server is lost, so will the UrBackup server in some cases)

PROBLEMS
a) “Server1’s” backup directory is much larger than I expected
b) The script copies ALL files in the “server1” backup directory which includes the “.directory_pool” folder. This folder contains over 1million files (600GB) so takes multiple days to copy to another media do to many small files
c) I do not have the option to setup an offsite UrBackup server as the building has a terrible upload speed.

QUESTIONS

  1. How can I trim the entire backup size so that I’m only retaining the following
    -1 full image (snapped monthly and replacing last month’s image)
    -1 full file copy (snapped monthly and replacing last month’s full file backup)
    -7 days of incremental file changes, taken hourly (up to 180)
  2. Exactly which directories MUST be copied to external media in order to restore a full file set or full linage from external media in the case of the building burning down?
  3. I’m operating under the assumption that if the client and urbackup server are lost together (fire, theft, etc) that I can setup a new urbackup server, import the database and backup set from the external drive, then boot a new server from the urbackup USB boot media and restore over the network. Is this correct/best?

I really appreciate the replies and patient answers, this seems like a good community and i’m surprised that more people aren’t “on” this. I was on the phone to my Veeam account rep today and he hadn’t even heard of UrBackup!

8

Hello, @Daniel_Pipe. Did you find the answers to those last questions? I would be interested to know. Thanks! (sorry for the necro thread)