I’m facing issues with full image backups either from Windows or Linux using VHD (Compressed) containers. Containers vary in size from 200 to 800 Gbs.
Images are failing during recovery phase with errors in logs:
Jan 17 20:28:26 apparel urbackupsrv[24205]: ERROR: Error while decompressing file (zstd). Error code Src size is incorrect
Jan 17 20:28:26 apparel urbackupsrv[24205]: ERROR: Error while decompressing file (zstd). Error code Src size is incorrect
Jan 17 20:28:26 apparel urbackupsrv[24205]: ERROR: Error while decompressing file (zstd). Error code Src size is incorrect
Jan 17 20:28:26 apparel urbackupsrv[24205]: ERROR: Error reading bitmap
Jan 17 20:28:26 apparel urbackupsrv[24205]: WARNING: Padding 4096 zero bytes during restore...
Jan 17 20:28:26 apparel urbackupsrv[24205]: ERROR: Error reading from VHD file during restore. Code 0
Jan 17 20:28:26 apparel urbackupsrv[24205]: Lost channel connection to ##restore##16412873686343. has_error=true
Then after few minutes:
Jan 17 20:31:58 apparel urbackupsrv[24205]: ERROR: Error while decompressing file (zstd). Error code Src size is incorrect
Jan 17 20:31:58 apparel urbackupsrv[24205]: ERROR: Error reading bitmap
Jan 17 20:31:58 apparel urbackupsrv[24205]: WARNING: Padding 4096 zero bytes during restore...
Jan 17 20:31:58 apparel urbackupsrv[24205]: ERROR: Error reading from VHD file during restore. Operation now in progress (code: 115)
And each few minutes above message gets repeated - seems like server is retying to read the file.
Failure happens at different restoration %% for different images, however same image fails at the same %%. For example different images might fail at 25% or at 60%, while if I will try to restore same image, it will permanently fail at let’s say 35%.
When failure occurs - restoration client just stays at failed %% forever. Disk activity halted and the only way to see that something is wrong is to look into server logs.
I store images on NAS and recover them by network to a testing VM. No errors reported on either file system of the NAS or network between NAS and testing VM. No CRC errors reported if I try to copy VHDZ container files on NAS directly. No errors reported by OS or Kernel (Centos 7). Filesystem there is ext4, which is setup over LVM JBOD.
Same NAS is being used by other services, including backup system - no issues are observed or reported there.
No errors reported during backup process as well. Not for Windows, not for Linux. They are considered successful by UrBackup server as well.
When I try to recover Incremental Image - I still get error in logs (likely during parent image recovery), however restoration process continues and finishes successfully. I’m not sure to which state system is recovered in such a case, however booting from recovered images works and I was unable to spot any FS corruption or missing files (I haven’t look hard though).
Thus questions:
- What additional information should I provide to help to pin Full image recovery issue?
- At which state system is recovered from Incremental image in this case? My assumption that such restoration should be corrupt. Do I need to perform more testing to confirm this or it doesn’t make sense?
Thank you.