Running UrBackup server 2.1.19 on debian with ZFS storage using incremental forever setup.
I have about 10 windows servers backing up to this server. I was testing my backups by restoring the images and booting them. 9 of the servers worked fine. The 10th server restore was successful using the 2.0.2 restore iso however, when I tried to boot it, the system went into recovery mode and nothing seemed to fix it.
I went back to my full backup and it restored and booted fine. I have a few weeks of incremental backups that I will go through to see if any of those boot. In the meantime, what could cause an incremental backup to not boot if the backup was successful and the restore was successful?
Usually it is dissimilar hardware. What combination is it?
Good point - This is a VM that was running on Hyper-V and is being restored to a KVM server for the tests. I will review the processor info.
This worked for the other 9 VMs though, so I am not sure what I will find.
Hyper-V has two configurable hardware generations, so maybe check that? Maybe you also find the actual error code during boot. Maybe use F8 to disable recovery or something.
It is a gen 1 guest on HyperV 2008.
The funny thing is, I can boot the full backup image without issue. I haven’t tested all of the incrementals but I tested the more recent ones and they won’t boot.
Then it might be a (serious) bug. To narrow down:
- Send me the logs on the server of the image backups of the client as well as the client log
- Check the image file on the server via
urbackupsrv internal --image_verify /path/to/image.raw . An error on the last block is ok.
- Reset CBT with
urbctctl.exe reset all then run an incremental image backup to check if CBT caused this problem
- Check if you can mount the image. If you can mount the image I can tell you a way to check which files are wrong.
Thanks for the help!
Edit: Sry image_verify instead of device_verify
I will send you the client logs. For the server logs, do you want /var/log/urbackup.log? I only have a couple of compressed versions of that if you need those too.
The image verify worked fine and I am able to mount the image.
When I try to reset CBT on the client I get the following:
C:\Program Files\UrBackup>urbctctl.exe reset all
Resetting change block tracking on volume E:… Cannot open volume. Last error:
Resetting change block tracking on volume C:… Cannot open volume. Last error:
Resetting change block tracking on volume D:… failed
Reseting change block tracking on volume D: failed. Error code: 1
Can you run the reset as admin?
The logs from the server web interface in the logs section. If you can narrow it down to the ones that are broken, then the ones around the time where that happend would be interesting. Mainly to check if CBT was enabled, or if something unusual happend.
/var/log/urbackup.log would be interesting if it was in debug logging mode at the time.
There is nothing interesting in the web interface logs on the server. The server log was not in debug mode so I don’t think we’ll get much from that.
I was able to successfully reset cbt as admin, thank you. I ran an incremental which took a long time as it was building its index. I then ran another incremental after that. Both of those incrementals boot just fine.
So does this point to something in cbt? Would the client log help you determine that?
Probably not. The best bet would be if there are some hints on how to reproduce the problem. For example one reason it can get corrupted is if the drive is written to without the driver being active. E.g. the Windows system recovery does this and if you mount the system disk as another disk in another VM.
I think I found the cause. Will do some tests then update ASAP.
Just checking in to see if there is anything I can provide to help out.
Sry, should be fixed with 2.1.17-cbt released last friday.
May I know if the updates have been commited to github?
Are you able to advise which cpp files were updated in order to fix this?