I have the need to restore an entire Linux client, because the disk has failed.
The server has a full full filebackup and some incremental backups. As the disk is broken, I have to download the backup and restore manually.
Unfortunately neither the full filebackup, nor the incremental backups can be downloaded. The server starts downloading, but leaves a corrupt download file, which cannot be opened.
The server logfile says:
2017-12-01 18:35:35: ERROR: Error while adding file “/mnt/urbackup/RaspberryPi1/171201-1538/root/usr/share/man/man3/Algorithm::Diff.3pm.gz” to ZIP file. Error: invalid filename. OS error: Code 0
2017-12-01 18:35:35: ERROR: Error while adding files and folders to ZIP archive
2017-12-01 18:40:04: ERROR: Error opening file “/mnt/urbackup/RaspberryPi1/171130-0935/root/opt/sonic-pi/app/server/native/raspberry/extra-ugens-wheezy/extra-ugens” for ZIP file download. Too many levels of symbolic links (code: 40)
2017-12-01 18:40:04: ERROR: Error while adding files and folders to ZIP archive
May be a limitation in the number of file a zip can have
Please try to restore using the client.
Put server-allow in client config , restart the client then start the restore from server
Hi,
many thanks for your response, however this is not possible. The broken disk is the system disk and cannot be restored via the gui. During restore, the os would fail
Is there a way to restore via gui to another disk?
Thanks a lot
Theoretically you should be able to restore over the running system (perhaps stop most applications first). Otherwise see the --map-to and --map-from parameters to urbackupclientctl restore.
I have tried restoring the os while it was running, but urbackup stopped at appoximately 30%.
Will habe a look at your advised option. Thanks a lot so far.
Yes, but on linux it s a pita to get the parameters right, like backup id, remember u need to use a server relative path and whatnot. You need to start the rstore from the web ui, look up the parameters in the url and use that.
I tried to write a tui for that, then tried to look up how it was done in the livecd tui, then gave up … It s way easier to use server side restore.
This is quite a challenging advise for me. Could not find any documentation, how to do it. The help function is unfortunately not comprehensive for me. Do you have an example for me please?
How to select the correct archive from the client side? How to exactly specify the path from/to?
The client is headless, no gui.
It replaces --map-from with --map-to while restoring. E.g. if you do not want to restore to the running system root, but to a mounted one --map-from / --map-to /mnt/root.
ok, got it, that simple. I’m in between deleting all incremental backups through the gui and will try a download the zip again, hoping that it works, because the are not that many links anymore. If this does not work, I will try as suggested by you with a mounted disk and map from - to.
This will take some time. Deleting is not that fast.
How can I assure, that Im not creating too many links with my backups? Where is the threshold? I was assuming that app. 4GB Backup of an os is not much for the system, but as it looks it has less to do with size but with number of files and number of incremental backups, correct?
Now I have deleted all previous incremental backups, just the full filebackup left. Even then I Can’t download the backup file :(. This function seems to be useless.
I have tried to restore now from the client to a mounted drive, using the following command: urbackupclientctl restore-start -b “last” -m / -t /mnt/restore/
The response is forever without doing anything
Starting restore. Waiting for backup server… /
The server log says:
2017-12-01 22:51:24: ERROR: Cannot read file metadata of file /mnt/urbackup/RaspberryPi1/171201-2032/root from /mnt/urbackup/RaspberryPi1/171201-2032/.hashes/root/.dir_metadata. Cannot start restore.
I’m starting to get frustrated and disapointed…It all looked so good, but when it comes to restore a full filebackup, it does’t seem to work.
past midnight, kinda sleep time for me.
If you want i can try to have a look at it around 1pm utc over teamviewer so you can keep an eye on what i do.
There should be no need to delete incrementals or so.
very kind of you.
I have setup a spare machine to try restoring.
At the moment I believe it doesn’t make much sense to accept your proposal, it would be wasted time for you, because the deletion of incremental backups obviously stumbled about the fact, that some backups had warnings (not errors) with regards to saving metadata.
At this point of time, I can neither download any backup (incremental/full) nor restore via client. It all leads to errors on the server.
It looks to me, that the harmless looking warnings during backup with regards to saving metadata don’t have an impact on restoring files or folders. But when I delete backups, the server cannot deal with that any more and messes up its database.
I will try to start over completely and restore the machine from another backup that I did without urbackup. Then I will delete the client in Urbackup and recreate it. Make 1 full and a few incremental backups and then switch to a spare machine in order to test restore. I will come back with the results and further questions.
I’d like to ensure, that I did not mess up urbackup by making mistakes. One question left for the moment: Can I ignore warnings with regards to metadata in the backups, yes or no?
Thanks a for your support.
There shouldn’t be any metadata errors/warnings, so please don’t ignore them. Could you post a few logs where it failed? Even if there are problems with metadata it should still have files in .hashes (with 2.1.x)…
In between I’m running 3 urbackup servers in parallel and some clients, investigating, what was going on.
Findings so far:
The initial problems with metada errors/warnings came from one client. The client had a problem with it’s filesystem (that’s why I wanted the restore and had all these questions).
The error was just with one directory which was showing these permissions/owners: d??? .gvfs
The backup terminated with “no access permissions”… I think we should take out this case from our discussion.
I’m currently recovering that machine from another backup and then I will continue with backup/restore tests vir urbackup again. I’M quite confident, that here the problems should not show up anymore.
With another machine, I had e.g. these error messages:
2017-11-26 19:01:26: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/pve/.dir_metadata. No such file or directory (code: 2)
2017-11-26 19:01:26: ERROR: Error opening metadata file for “dclientdl0/etc/pve”
2017-11-26 19:01:26: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/python/debian_config. No such file or directory (code: 2)
2017-11-26 19:01:26: ERROR: Error opening metadata file for “fclientdl0/etc/python/debian_config”
2017-11-26 19:01:26: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/python/.dir_metadata. No such file or directory (code: 2)
2017-11-26 19:01:26: ERROR: Error opening metadata file for “dclientdl0/etc/python”
2017-11-26 19:01:26: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/python2.7/sitecustomize.py. No such file or directory (code: 2)
2017-11-26 19:01:26: ERROR: Error opening metadata file for “fclientdl0/etc/python2.7/sitecustomize.py”
2017-11-26 19:01:26: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/python2.7/.dir_metadata. No such file or directory (code: 2)
2017-11-26 19:01:26: ERROR: Error opening metadata file for “dclientdl0/etc/python2.7”
2017-11-26 19:01:26: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/python3/debian_config. No such file or directory (code: 2)
2017-11-26 19:01:26: ERROR: Error opening metadata file for “fclientdl0/etc/python3/debian_config”
2017-11-26 19:01:26: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/python3/.dir_metadata. No such file or directory (code: 2)
2017-11-26 19:01:26: ERROR: Error opening metadata file for “dclientdl0/etc/python3”
2017-11-26 19:01:26: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/python3.5/sitecustomize.py. No such file or directory (code: 2)
+++++++++++++++++++++++++++++++++
2017-11-26 20:27:53: ERROR: Error opening metadata file for “fclientdl0/ssl/certs/Verisign_Class_1_Public_Primary_Certification_Authority.pem”
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/Verisign_Class_1_Public_Primary_Certification_Authority_-G3.pem. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for "fclientdl0/ssl/certs/Verisign_Class_1_Public_Primary_Certification_Authority-G3.pem"
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/Verisign_Class_2_Public_Primary_Certification_Authority-G2.pem. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for "fclientdl0/ssl/certs/Verisign_Class_2_Public_Primary_Certification_Authority-G2.pem"
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/Verisign_Class_2_Public_Primary_Certification_Authority-G3.pem. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for "fclientdl0/ssl/certs/Verisign_Class_2_Public_Primary_Certification_Authority-G3.pem"
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority.pem. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for “fclientdl0/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority.pem”
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority-G3.pem. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for "fclientdl0/ssl/certs/Verisign_Class_3_Public_Primary_Certification_Authority-_G3.pem"
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/Visa_eCommerce_Root.pem. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for “fclientdl0/ssl/certs/Visa_eCommerce_Root.pem”
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/WellsSecure_Public_Root_Certificate_Authority.pem. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for “fclientdl0/ssl/certs/WellsSecure_Public_Root_Certificate_Authority.pem”
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/XRamp_Global_CA_Root.pem. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for “fclientdl0/ssl/certs/XRamp_Global_CA_Root.pem”
2017-11-26 20:27:53: ERROR: Cannot read from metadata file /media/bwg/SSD/urbackup-neu/bwgpve/171126-1650/.hashes/root/etc/ssl/certs/a0bc6fbb.0. Code 0
2017-11-26 20:27:53: ERROR: Error opening metadata file for “fclientdl0/ssl/certs/a0bc6fbb.0”
Sorry, that just shows that the files are missing. The question would be why… e.g. cd to the .hashes directory in different backups and look at where the symlinks are wrong etc. and then look at the logs of the backups (and system logs) to get a clue what went wrong.
ok. Let’s analyze step by step, that is reproducable.
I have uninstalled a client by the following commands:
/usr/local/sbin/uninstall_urbackupclient
rm -Rf /usr/local/var/urbackup
Then I have installed a new client version 2.1.16 with option no. 4 on two clients with partitions to backup, vfat/Fat32 and ext4 and started two times a full backup:
The result is twice the same. Ignoring the warning with regards to metadata will later on cause problems with incremental backs and restores, correct?
Exactly the same version 2.1.16 on two other client runs to backup ext4 / lvm partions without any errors/warnings.
2017-12-03 10:20:58: WARNING: Error getting complete file “rnrRXDh7jzItwS67ZqwI|root/usr/local/var/urbackup/data/filelist_new_0.ub” from RaspberryPi2. Errorcode: CANNOT_OPEN_FILE (3)
2017-12-03 10:42:27: WARNING: Not all folder metadata could be applied. Metadata was inconsistent.
2017-12-03 12:08:08: WARNING: Error getting complete file “rnrRXDh7jzItwS67ZqwI|root/usr/local/var/urbackup/data/filelist_new_0.ub” from RaspberryPi2. Errorcode: CANNOT_OPEN_FILE (3)
2017-12-03 12:22:19: WARNING: Not all folder metadata could be applied. Metadata was inconsistent.
Maybe that urbackup cannot deal with fat32 partions?
No this warning is just about the server not being able to set the last modified times of the backup files themselves. I haven’t gotten around to looking at this problem, but that doesn’t cause restores to fail or the metadata displayed on the web interface to be incorrect. So this does not reproduce the problem.
I’m a bit worried to see any warnings in the log, especially when I don’t understand their meaning.
The question is, what do I need to do get rid of these warnings. Two hardware different clients backup without any warning and the two identical machines show exactly the same warnings.
I’m more than happy to run more tests to help fixing the problem, but I need your advice, what I should do.
Thanks a lot for your continued support.
