Problems Restoring Image To Same Size HD

Server: v 2.4.13
Windows Client: v. 2.4.10

I am testing a vhdz image restore, but receive error that new drive is too small:

“The disk you are trying to restore to is too small. The partition to be restored has size 918.431 GB, but with this disk only 918.431 GB are available.”

These drives are same brand name and size. Must a new drive always be larger? I usually choke a new TB drive down to ~500GB partition to avoid always having a larger drive available as a backup, but these clients came preinstalled. Someday I am going to need a 20TB drive just to restore 50GB of data!

So, I put in a 2TB drive to restore a 1TB image. Same message.?!

Can you have a look at the restore logfiles?

Ctrl+Alt+F2 to switch to a console. Then

sudo su
cd /root
cat restore_mode.txt

Quite a bit of info there, but I think the important part is:

Imagesize 986157678080
Used bytes 986157678592
Error: Could not write pipe! DownloadImage-3 size 5792 off 0

I see that the restore process is actually creating the partitions (as I expected), but went ahead and used fdisk to delete all partitions and create just one large 1.8TB partition. The restore created its own table with a partition too small. ??

Could you upload the .mbr file of the partition here? (it just contains the disk MBR/GPT)

And perhaps also take a look at /root/restore_wizard.txt

I added .txt to get it to upload.

Image_C_201104-1153.vhdz.mbr.txt (33.6 KB)

Restore_wizard.txt indicates the same sizing info that’s in my first post here.
Too small.

Thanks for your help!

Please post the exact error message.

Sorry. Here is the majority of it:

2020-11-05 14:07:00: Login Response: err – err
2020-11-05 14:07:00: ERROR: Error during login: err – err
2020-11-05 14:07:07: Login Response: ok
2020-11-05 14:07:30: Restoring to mbr.dat
2020-11-05 14:07:30: Logical block size: 512
2020-11-05 14:07:30: Selected device: /dev/sda Partition: 3
2020-11-05 14:07:41: Restoring to /dev/sda3
2020-11-05 14:07:41: ERROR: Output device too small. File size = 986157416448 needed = 986157678080
2020-11-05 14:10:07: Restoring to mbr.dat
2020-11-05 14:10:07: Logical block size: 512
2020-11-05 14:10:07: Selected device: /dev/sda Partition: 3
2020-11-05 14:10:17: Restoring to /dev/sda3
2020-11-05 14:10:18: ERROR: Output device too small. File size = 986157416448 needed = 986157678080
2020-11-05 14:10:26: Restoring to /dev/sda3
2020-11-05 14:10:26: ERROR: Output device too small. File size = 986157416448 needed = 986157678080

Thanks! Interesting. So it is 511 sectors (each 512 bytes) too short.

It uses Number of sectors * sector size from fsutil fsInfo ntfsInfo C: as volume size. So that would be interesting (but I guess you don’t have it booting?).

Short term workaround may be using https://urbackup.atlassian.net/wiki/spaces/US/pages/2981890/How+to+restore+via+command+line .

Long term, I guess, the best solution would be to have a check if the partition fits onto the disk and then ignore anything that cannot be restored…

Is this somehow related to the vhdz format? I am normally using Raw copy-on-write, but for testing, I put one of these clients in a new group and changed the image format to vhdz in an attempt to make sure I can use this type at a different location (no btrfs there). The image backup appears to go fine, but cannot restore as noted above.

I took a second client (one that I built) and moved it to the new test group and initiated a full image backup. The backup seemed to go fine, but the restore hit the same error. I then tried one of the COW backups from this client and restore completed without errors and test computer boots up as the image provided.

Would the new full vhdz image (with a previously used COW image settings) work on it’s own?

Thanks,

Paul

If I backup a full image using just vhd, I get an error, also. Drive not big enough.

No, the actual partition (1926088704 sectors) is smaller than what Windows says it is (1926089215 sectors). That causes the issue.

Running that would confirm it.

This comes from the drive that the test image is sourced from:

C:\Windows\system32>fsutil fsinfo ntfsInfo c:
NTFS Volume Serial Number : 0xe8e64b93e44b03f0
Version : 3.1
Number Sectors : 0x000000003a352fff
Total Clusters : 0x000000000746a5ff
Free Clusters : 0x000000000625c6ba
Total Reserved : 0x0000000000002c70
Bytes Per Sector : 512
Bytes Per Physical Sector : 4096
Bytes Per Cluster : 4096
Bytes Per FileRecord Segment : 1024
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 0x00000000111c0000
Mft Start Lcn : 0x00000000000c0000
Mft2 Start Lcn : 0x0000000000000002
Mft Zone Start : 0x0000000000e9ca60
Mft Zone End : 0x0000000000e9e760

Is this somehow a Microsoft Windows issue?

After testing (and failing) on two Windows clients, I am guessing this is not just a fluke. The Raw copy-on-write format works in testing, but both VHD and compressed VHD cannot be restored via GUI. Although I’m sure I can get a drive restored via command line during testing, I don’t want to rely on that when I am under pressure in non-testing situations.

What can I do to make the VHD format work via GUI?

Weird, that’s about 500GB, but you partition table:

Disk /dev/loop0: 5859375000 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 6094E888-08B0-4B40-8347-732ADE0BB0BB
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 15725 sectors (7.7 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          534527   260.0 MiB   EF00  EFI system partition
   2          534528          567295   16.0 MiB    0C01  Microsoft reserved ...
   3          567296      1926655999   918.4 GiB   0700  Basic data partition
   4      1926656000      1928663039   980.0 MiB   2700  Basic data partition
   5      1928663040      1953511423   11.8 GiB    0700  Basic data partition

Command (? for help): i
Partition number (1-5): 3
Partition GUID code: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data)
Partition unique GUID: 7FF385CF-85B3-4E91-BC2B-E7BC5008F343
First sector: 567296 (at 277.0 MiB)
Last sector: 1926655999 (at 918.7 GiB)
Partition size: 1926088704 sectors (918.4 GiB)
Attribute flags: 0000000000000000
Partition name: 'Basic data partition'

And

Say something different.

Sorry - I mixed up the posted info because I started testing on a second client that only has a 500GB drive. This is from the client I initially started this thread about:

C:\WINDOWS\system32>fsutil fsInfo ntfsInfo C:
NTFS Volume Serial Number : 0x50ecfca5ecfc870e
NTFS Version : 3.1
LFS Version : 2.0
Total Sectors : 1,926,088,703 (918.4 GB)
Total Clusters : 240,761,087 (918.4 GB)
Free Clusters : 226,846,080 (865.3 GB)
Total Reserved Clusters : 60,728 (237.2 MB)
Reserved For Storage Reserve : 0 ( 0.0 KB)
Bytes Per Sector : 512
Bytes Per Physical Sector : 4096
Bytes Per Cluster : 4096
Bytes Per FileRecord Segment : 1024
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 552.75 MB
Mft Start Lcn : 0x00000000000c0000
Mft2 Start Lcn : 0x0000000000000002
Mft Zone Start : 0x000000000127bb00
Mft Zone End : 0x0000000001288320
MFT Zone Size : 200.13 MB
Max Device Trim Extent Count : 0
Max Device Trim Byte Count : 0
Max Volume Trim Extent Count : 62
Max Volume Trim Byte Count : 0x40000000

Could you have a look at backup_server.db on the backup server (with e.g. SQLite browser) and run

SELECT path, version, clientid, letter FROM backup_images;

and post the output/send that to me. Thanks!

Ok, so the bug might be that it accidentally sets version in backup_images to zero if you have a connected client, then set the image backup format to cow-raw and then back to vhd(z). After setting it back and forth it will create images with version=0. And those don’t fit onto the partition (and the wrong data gets restored if they would fit on partition).

Workarounds:

Fix the existing images: UPDATE backup_images SET version=1. Restart the server each time you set the image backup format from vhd <-> raw.

Thanks for all your help.

path version clientid letter
urbackup/test01/201104-1152_Image_SYSVOL/Image_SYSVOL_201104-1152.vhdz 0 12 SYSVOL
/urbackup/test01/201104-1152_Image_ESP/Image_ESP_201104-1152.vhdz 0 12 ESP
/urbackup/test01/201104-1153_Image_C/Image_C_201104-1153.vhdz 0 12 C:
/urbackup/test01/201104-1234_Image_D/Image_D_201104-1234.vhdz 0 12 D:
/urbackup/test02/201106-1324_Image_SYSVOL/Image_SYSVOL_201106-1324.vhdz 0 38 SYSVOL
/urbackup/test02/201106-1324_Image_C/Image_C_201106-1324.vhdz 0 38 C:
/urbackup/test02/201106-1433_Image_SYSVOL/Image_SYSVOL_201106-1433.vhd 0 38 SYSVOL
/urbackup/test02/201106-1434_Image_C/Image_C_201106-1434.vhd 0 38 C:
/urbackup/sp03/201109-1056_Image_SYSVOL/Image_SYSVOL_201109-1056.raw 0 4 SYSVOL
/urbackup/sp03/201109-1056_Image_ESP/Image_ESP_201109-1056.raw 0 4 ESP
/urbackup/sp03/201109-1056_Image_C/Image_C_201109-1056.raw 0 4 C:

Clientid 12 was my first test, clientid 38 was my second, 500GB, test. And I through in clientid 4 which is a normal COW image.

OK - There is progress. I am not too worried about existing backups from the test, so, once again, I placed my test clients in the test group, checked that the image format was vhdz, stopped the server and started the server. The version on my new test client’s full image backups now indicate 1, so I felt better.

Attempting to restore a new image started out without sizing error. Restore status indicated “0%” for about 15 minutes for the ~50GB partition and then the status went immediately to 100%. Watching the server GUI activity tab during this time indicated that the restore required time was 2 minutes. After the client’s restore screen came to 100%, five minutes later the bottom of the screen indicated: “WARNING: Read Timeout: Retrying”. Client indicates no more new messages (yet).

The server log entry:

2020-11-11 07:28:38: ERROR: Writing to output pipe failed processMsg-1

Client restore_wizard.txt:

2020-11-11 07:26:35: Login Response: err
2020-11-11 07:26:35: ERROR: Error during login: err
2020-11-11 07:26:45: Login Response: ok
2020-11-11 07:27:39: Restoring to mbr.dat
2020-11-11 07:27:39: Selected device: /dev/sda Partition: 2
2020-11-11 07:27:49: Restoring to /dev/sda2
2020-11-11 07:43:33: WARNING: Read Timeout: Retrying

And client restore_mode.txt:

2020-11-11 07:26:45: rc=0 hasError=false state=3
2020-11-11 07:26:45: rc=0 hasError=true state=0
2020-11-11 07:26:56: ClientService cmd: GET BACKUPIMAGES
2020-11-11 07:26:56: Waiting for pings…
2020-11-11 07:26:56: done. (Waiting for pings)
2020-11-11 07:26:56: rc=0 hasError=true state=0
2020-11-11 07:27:05: ClientService cmd: PONG
2020-11-11 07:27:15: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=0&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:27:15: New channel: Number of Channels: 2
2020-11-11 07:27:39: urbackupserver: No available slots… starting new Worker
2020-11-11 07:27:39: ClientService cmd: DOWNLOAD IMAGE#pw=O5DbOLANpgegh7Xw5LbGInU6RDKQ6e&img_id=637819961&time=1605058097&mbr=true
2020-11-11 07:27:39: Downloading image…
2020-11-11 07:27:39: Waiting for pings…
2020-11-11 07:27:39: done. (Waiting for pings)
2020-11-11 07:27:39: In mutex…
2020-11-11 07:27:39: Downloading from channel 0
2020-11-11 07:27:39: urbackupserver: No available slots… starting new Worker
2020-11-11 07:27:39: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:27:39: Imagesize 556
2020-11-11 07:27:39: rc=0 hasError=true state=0
2020-11-11 07:27:39: Downloading MBR…
2020-11-11 07:27:39: Downloading MBR done
2020-11-11 07:27:39: Download done -2
2020-11-11 07:27:47: Internet mode not enabled
2020-11-11 07:27:49: ClientService cmd: DOWNLOAD IMAGE#pw=O5DbOLANpgegh7Xw5LbGInU6RDKQ6e&img_id=637819961&time=1605058097&mbr=false
2020-11-11 07:27:49: Downloading image…
2020-11-11 07:27:49: Waiting for pings…
2020-11-11 07:27:49: done. (Waiting for pings)
2020-11-11 07:27:49: In mutex…
2020-11-11 07:27:49: Downloading from channel 0
2020-11-11 07:27:49: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:27:49: urbackupserver: No available slots… starting new Worker
2020-11-11 07:27:49: ClientService cmd: GET DOWNLOADPROGRESS#pw=O5DbOLANpgegh7Xw5LbGInU6RDKQ6e
2020-11-11 07:27:49: Sending progress…
2020-11-11 07:27:49: Imagesize 500000882176
2020-11-11 07:27:49: Used bytes 500000882688
2020-11-11 07:27:59: Client timeout in ClientConnector::Run
2020-11-11 07:28:00: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:28:10: Client timeout in ClientConnector::Run
2020-11-11 07:28:11: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:28:15: ClientService cmd: PONG
2020-11-11 07:28:21: Client timeout in ClientConnector::Run
2020-11-11 07:28:22: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:28:32: Client timeout in ClientConnector::Run
2020-11-11 07:28:33: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:28:37: rc=0 hasError=true state=3
2020-11-11 07:28:41: rc=0 hasError=true state=0
2020-11-11 07:28:41: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:28:51: Client timeout in ClientConnector::Run
2020-11-11 07:28:52: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:29:02: Client timeout in ClientConnector::Run
2020-11-11 07:29:03: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:29:13: Client timeout in ClientConnector::Run
2020-11-11 07:29:14: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL
[many repeats every 10 seconds]
capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:40:25: Client timeout in ClientConnector::Run
2020-11-11 07:40:26: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:40:36: Client timeout in ClientConnector::Run
2020-11-11 07:40:37: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:40:43: ERROR: Read Timeout -2 CS
2020-11-11 07:40:43: Download done -2
2020-11-11 07:40:43: Client timeout in ClientConnector::Run - Channel (total timeout)
2020-11-11 07:40:43: ClientService cmd: GET DOWNLOADPROGRESS#pw=O5DbOLANpgegh7Xw5LbGInU6RDKQ6e
2020-11-11 07:40:43: Sending progress…
2020-11-11 07:40:47: Client timeout in ClientConnector::Run
2020-11-11 07:40:48: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:40:48: New channel: Number of Channels: 1
2020-11-11 07:41:48: ClientService cmd: PONG
2020-11-11 07:42:48: ClientService cmd: PONG
2020-11-11 07:42:48: rc=0 hasError=true state=3
2020-11-11 07:42:48: ClientService cmd: #IgsZlIMHXFghhDSq6wQOk#1CHANNEL capa=142649&token=ZnaOhyBmhx4DSaBLhgDF&restore_version=1&virtual_client=
2020-11-11 07:42:48: New channel: Number of Channels: 1
2020-11-11 07:43:48: ClientService cmd: PONG
2020-11-11 07:44:48: ClientService cmd: PONG
2020-11-11 07:45:48: ClientService cmd: PONG

This is now where I am stuck.