Restore (download as zip) problem

Server 2.0.33, web interface use “restore files as zip”, downloading cause web losing response.
downloaded file is broken.
WinRAR shows the zip file has infinite loop of dir in C:
Further look shows that the “AppData” folder, “Users” folder and “Document and Settings” folder caused problem: 1) server loses response; 2) downloaded zip file is broken

same here.

UrBackup 2.0.31, windows client backed up whole drive C as files and I want to get them.

Downloading zips renders in unreadable files. Unzip on mac and linux complains that the file is a part of a bigger archive and does not extract anything.

I’ve tried rsyncing the data to my local PC, but again I got lost in the symlink loops. The problem here is that symlinks are used by UrBackup for two things: linking from “current” to the previous version of the file if it does not need changing AND copying NTFS symlinks that Windows makes itself. For some reason links to oneself pop up and rsync cannot properly handle them.

Frustrating, as I’ll have to retrieve a whole windows PC file by file manually.

OK, i’ve managed to get some testing done, and it looks like I’ll be able to get all the files in a nice manner.

This is applicable IF someone backs up windows whole drives and/or home directories of windows users with looped symlinks.

  • Go to the home of the latest incremental file backup on the urbackup server. The directory structure will be like:

$URBACKUP_PREFIX/$CLIENT_NAME/current

  • find loops:

find -L . | grep "loop "

  • copy the structures you need somewhere else (you’ll be deleting the loops there) (the command below copies symlinks just like that)

copy -r . /SOMEWHERE_ELSE

  • go to SOMEWHERE_ELSE and delete the looped symlinks

now you have a structure with symlinks to previous versions of files, that can be zipped and moved somewhere else.

Could you post a list of symlink loops? Are they present in the backup source as well?

So I’ve got a laptop backed up with everything from drive C: and this is what I get:

find -L . | grep "loop "

find: File system loop detected; ‘./C_0/ProgramData/Dane aplikacji’ is part of the same file system loop as ‘./C_0/ProgramData’.
find: File system loop detected; ‘./C_0/Documents and Settings/All Users/Dane aplikacji’ is part of the same file system loop as ‘./C_0/Documents and Settings/All Users’.
find: File system loop detected; ‘./C_0/Documents and Settings/username/AppData/Local/Dane aplikacji’ is part of the same file system loop as ‘./C_0/Documents and Settings/username/AppData/Local’.
find: File system loop detected; ‘./C_0/Documents and Settings/username/Ustawienia lokalne/Dane aplikacji’ is part of the same file system loop as ‘./C_0/Documents and Settings/username/Ustawienia lokalne’.
find: File system loop detected; ‘./C_0/Users/All Users/Dane aplikacji’ is part of the same file system loop as ‘./C_0/Users/All Users’.
find: File system loop detected; ‘./C_0/Users/username/AppData/Local/Dane aplikacji’ is part of the same file system loop as ‘./C_0/Users/username/AppData/Local’.
find: File system loop detected; ‘./C_0/Users/username/Ustawienia lokalne/Dane aplikacji’ is part of the same file system loop as ‘./C_0/Users/username/Ustawienia lokalne’.

As you can see some dir names are polish. Dane aplikacji means Application Data, Ustawienia lokalne = local settings, and so on.

I think that this happens due to the way windows manages links between Documents and Settings and Users. The only way to manage this properly if links are used to point to old file versions is to write them down in a separate file when backing up and create them again when restoring from backup. Downloading ZIPs of C_0 results in corrupted files.

I wonder how will it work when I will try to download the data onto the client using urbackup. Will it blow up?

I need to restore a PC that I had backed up and I will restore it from the image backup and then get the freshest files on it from file backup. I will keep you all posted.

Hello *,

I know that every one thinks that they found the problem with the web-zip download, but after reading all the posts related and making some tests on my own, because this happened to me too, I think that I found the problem.

It is not because of the file size (most cases are below 4GB), not because of links or file names in the archive, but because of the number of the files.

When it passes 65536 files it gives me this error:
2016-11-07 21:32:44: ERROR: Error while adding file “/opt/urbackup/user1/161107-1058/WinMent/DATA/AQERT/2013_08/DISTRIB.XG0” to ZIP file. RC=0
2016-11-07 21:32:44: ERROR: Error while adding file “/opt/urbackup/user1/161107-1058/WinMent/DATA/AQERT/2013_09” to ZIP file. RC=0
2016-11-07 21:32:44: ERROR: Error while adding files and folders to ZIP archive

The folder “/opt/urbackup/user1/161107-1058/WinMent/DATA/AQERT/2013_08/” does contain more files after DISTRIB.XG0 but it stops at this one (because this is file number 64k in the archive) , it tries the next subfolder … without luck, and after that exits with corrupted archive.

The error can happen on other folders to, but it stops every time at the file with number 64k.

I use the latest version 2.0.36 but the problem is still there.

Uroni … please help us. :slight_smile:

Thank you

I have the same or similar problem.

I have UrBackup 2.0.36 installed on Windows Server and I can´t download as zip large clients, the download is always cut over in 5gb.

What I can do?

Thank you.

Please use some other way to restore big folders/folders with a lot of files. You can restore via the client or e.g. by sharing the backup via samba/Windows file sharing.

I´m sorry but i don´t understand you.

Restore via the client? The restore has launched from managed url, and the restore options to offer this page is download the directories in zip archive.

The other option, restore by sharing backup folder caused new problems, the windows explorer not reconiced symbolics links and not is posible restore multiple folders.

Regards.

See also chapter 9 https://www.urbackup.org/administration_manual.html#x1-800009 in the manual.

on the client run :
urbackupclientctl restore-start --help

It´s possible run this command in a windows client?

I have done a high backup with UrBackup but the pc of this client is out of order.

The situation now is the next one:

PC1-Client1 out of order so I have installed a new client of UrBackup and from this client I try to download the documents from backup of PC1 but fails because of the high size or the big number of files.

Can I overturn all the data from backup of PC1 to PC2?

I can t be 100% sure the command exists on windows, but given how urbackup works,i am pretty sure it does
look for the exe in c program urbackup , or something like that

If you try like this, using the agent client instead of the zip file, it still fails ?

I have reviewed the installation of several windows clients and I do not see the restoration command.

“C:\Program Files\UrBackup\UrBackupClient_cmd.exe” restore-start --help

But you ll have to read the options

Thank you. The command in windows is as you indicate in your last comment. Now I will check the options to see if I can start the recovery.

Regards.

Hello,

Our setup has to back up a large number of small files.
A zipped restore is around 27MB but contains around 100k files.
Can I help debugging/rewriting the code ?
Will you merge it back to your code ?

Thank you.

The problem is that this library https://github.com/richgel999/miniz did not get zip64 support as promised at https://richg42.blogspot.de/2014/03/zip64-version-of-miniz-library-released.html . It seems it needs a new maintainer https://twitter.com/richgel999/status/552157126529785856 .

The alternative would be switching to e.g. libarchive , but that is a larger dependency plus I feel like the build in restore plus the above other options should be enough.
Btw. I think restoring with the build in restore should be faster than zip even with small files.

Hmm, https://github.com/mdsitton/miniz/tree/vogl-changes looks promising. We’ll probably have to run some tests, though.