Hi there, I haven’t been able to find the option in the settings, but is there simple way or option to unfollow and disable backing up symlink destinations? I’m currently testing urBackup server 1.4.4 and client 1.4.3 on Debian 7.7. I know that I can exclude certain directories, however I would like the choice to not backup/follow symlink destinations.
Hi there, thanks for the reply. Ideally I would have liked the simple setting to turn on or off the following/backing up of symlinks. I don’t really need any folder granularity setting to be honest. For the time being, I have found a way around this by adding exclusions to certain folders containing symlinks. This is the only thing that has led me back to using BackupPC for the office. urBackup is a great bit of software and I’ve started using it at home, but due to having multiple servers with shared user spaces in the office, I have many symlinks pointing to some large folders which end up being duplicated when backed up.
This is a long post but I hope it will be beneficial so please bear with me. A quick overview of what I’ve covered below:
Infinite symlink loops: what they look like, example of how Wine symlinks cause them;
Creating a script: description of a script used to investigate and classify symlinks that will be encountered during a UrBackup run;
"Hooking" the script: forcing UrBackup to terminate the backup if a non-excluded, infinite symlink loop is identified;
Script output: links x2 to pastebin.ubuntu.com, showing how it looks (the output of the script, that is);
Script run time: it’s pretty snappy and can be made even snappier;
Linux adoption of UrBackup: why Wine needs special attention if UrBackup is going to be successful working with Linux clients;
Linux backups - include symlinks or not?: the real meat of this issue, bringing some arguments and alternatives (based on BackupPC) that suggest that symlinks should not be followed when backing up Linux clients;
Summary: since this is a ridiculously long posting, I want to make it as easy as possible for you to reply, if you’re still there!
Infinite symlink loops
Had a problem where indexing wouldn’t complete and the file list had reached 300GB after 2 hours!
Uroni said that it looked like infinite symlink loops and to test with tail -f /var/log/urbackup_client.log. Looking into the log, this is just one example of what I found:
ERROR: No permission to stat "/root/.wine/drive_c/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/c:/users/root/My Documents/.wine/dosdevices/z:/home/abcde/.byobu/.ssh-agent"
A slightly convoluted path to the .ssh-agent file, then!
Looked at Wine symlinks a bit further (a sample user’s directory):
Creating a script to identify potential infinite symlink loops
With numerous computers to backup, needed an automated solution. Have managed to put together a shell script (#!/bin/sh) that does the following:
Takes input values for DEFAULT_DIRS and EXCLUDE_FILES in exactly the same format as typed in the UrBackup web interface
Works through each DEFAULT_DIR:
Uses find ${DEFAULT_DIR} -type l to locate all symlinks under that directory; and then with each symlink:
Captures the symlink target
Compares the symlink target with each of the DEFAULT_DIRS, to establish if it is:
Inside: target is inside at least one of the DEFAULT_DIRS - harmless
Outside: target is outside all of the DEFAULT_DIRS - potentially troublesome
Infinite symlink loop: target is an ancestor to at least one of the DEFAULT_DIRS - the backup must not proceed unless resolved by an exclusion rule
The symlink itself is compared to each of the EXCLUDE_FILES to see if it is covered by one of the rules
All of the information is presented using column for formatting - making it easier to see the state of affairs and what needs to be done
Exit message and return code are based on the the number of infinite symlink loops that have not been covered by an exclusion rule
“Hooking” the script into UrBackup to prevent backups where applicable
While useful to be able to manually run to check existing/proposed configurations, really needed this to run before each backup, especially where settings may have been changed before a manual check could be run. Was able to force this in the following way:
Found that the very latest values for DEFAULT_DIRS and EXCLUDE_FILES are available on the client in /usr/local/var/urbackup/data/settings.cfg - improved the script to extract these if called without any arguments
Used the script as /etc/urbackup/prefilebackup so run before indexing/backup
Upon infinite symlink loop error, backup is not allowed to proceed by forcibly killing the UrBackup client and then restarting it after a 5 second delay (this really needs a better solution)
Web interface log shows the error as Constructing of filelist of "{client}" failed - TIMEOUT(1)
Script output is appended to /var/log/urbackup_client.log so that an admin can see what exclusions need to be introduced to get the backup working (since backup wouldn’t work anyway, without the necessary changes)
Here running /etc/urbackup/prefilebackupwithout arguments, i.e. this would be run via. settings propagated to the client from the UrBackup server
Have ensured that the most important output is right at the bottom - identification of the potential infinite symlink loops and whether they have been resolved via. exclusions
In this case, the exclusion rules */.wine*/dosdevices and */.wine*/drive_c/users/*/My Documents have been sufficient to cover the 3 problems
Conclusion: The backup must be prevented until the infinite symlink loop(s) are resolved.
Here running /etc/urbackup/prefilebackupwith arguments for both DEFAULT_DIRS and EXCLUDE_FILES, i.e. running the script manually with test scenarios to see if exclusion rules are sufficient
In this case, have used the same settings as the first run, except have removed */.wine*/dosdevices from the exclusion rules, to see what would happen
Again, looking at the most important output at the bottom, can see that removing the rule has caused 2 of the infinite symlink loops to become “active”
If these were the actual settings on the server, the client would terminate the backup by resetting itself (which to reiterate, might not be a very appropriate way of going about this)
Also note:
The output also displays “outside” symlink targets, the grouping in the middle, which is shown as SYMLINK (MAYBE EXCLUDE: LINK_TARGET outside of backup roots)
These could make it much easier to formulate the exclusion rules, to eradicate issues such as:
How long does the script take to run?
More in seconds, rather than minutes - generally, will be much shorter than the indexing phase of the backup
Considering it saves hours, then probably not a bad trade-off
Running manually from the terminal, can see the progress, to get an idea of how long it would take with different sets of input DEFAULT_DIRS and EXCLUDE_FILES
Many types of filtering available within the script for tweaking; e.g.
ignoring symlinks that point to files makes it much quicker;
can ignore entire directory trees where no concerns are present, such as /etc;
etc., etc. (pun intended)
Linux adoption of UrBackup: Wine symlinks will be problematic
Wine [https://www.winehq.org/] is a very common package installed by many Linux users. By default, it makes multiple symlinks as shown further above. Some of these will definitely cause infinite symlink loops, when backing up user home directories.
I have found that the following exclusion pattern pretty much works for all users’ .wine directories on a system:
The additional asterisk in .wine* is to (optimistically) cover all of the wineprefixes [http://wiki.winehq.org/BottleSpec] that users will make - but this is only as good as users sticking to the .wine... directory naming prefix, for all of their wineprefixes!
Ultimately, having to manually add these exceptions is going to lead to more users with problems. Really needs an automated solution of some kind.
Linux backups - include symlinks or not?
With the current system, every change made to the configuration could arbitrarily introduce infinite symlink loops to backups. All it takes is shortening one of the paths in DEFAULT_DIRS or removing the wrong rule from EXCLUDE_FILES. At the very least, “outside” symlinks (see above) could lead to inappropriate directories being backed up - an undesirable outcome.
Having come from BackupPC myself, knowing that I had no such issues with symlinks when using it, I did a little research to understand how it dealt with symlinks.
… In fact, by default samba makes symbolic links look like the file or directory that they point to, so you could get an infinite loop if a symbolic link points to the current or parent directory. If you really need to use Samba shares for linux/unix backups you should turn off the “follow symlinks” samba config setting. …)
Similar to our situation; here, turning off “follow symlinks” prevents infinite loops - not that I used Samba, either…
Linux/unix file systems support several special file types: symbolic links, character and block device files, fifos (pipes) and unix-domain sockets. All except unix-domain sockets are supported by BackupPC (there’s no point in backing up or restoring unix-domain sockets since they only have meaning after a process creates them). Symbolic links are stored as a plain file whose contents are the contents of the link (not the file it points to). This file is compressed and pooled like any normal file. Character and block device files are also stored as plain files, whose contents are two integers separated by a comma; the numbers are the major and minor device number. These files are compressed and pooled like any normal file. Fifo files are stored as empty plain files (which are not pooled since they have zero size). In all cases, the original file type is stored in the attrib file so it can be correctly restored.
Emphasis above is mine
This seems to be the clearest explanation of how BackupPC works with symlinks; is there any merit in adopting this method for UrBackup as well?
Me too! But with a bit more besides: as a default setting on Linux, I would prefer not to follow symlinks.
Default: so that I don’t have to remember to turn this off when providing a UrBackup server for multiple Linux clients; please save me from major embarrassment when I have to explain to my client that all of their backups have failed because I forgot to set the little thingummy setting…!
Not: The symlinks’ targets themselves; however, similar to BackupPC, it would be brilliant if each symlinks’ literal contents could be backed up (and be restorable)
Personally, following symlinks is not a feature that is critical - since I wouldn’t be doing my job properly if I didn’t explicitly backup all of the important directories in the first place.
As for granularity, this might be much more complexity for very little gain. If the admin user defines the settings wrongly, he still isn’t immune to all of the potential banana skins. Not to mention the amount of headache involved in trying to work out these settings in the first place (for every single client?!) and then maintain them all - too much micro-management.
Summary: Discussion points / questions
Replying to the above is a waste of time… uh, I mean to say that it will waste your time referencing the issue you’re replying to! So, in a bid to make it as easy as possible for anyone out there who cares to respond, perhaps a numbered list would make things a little easier:
@Uroni@mhohr@chungyan5@AnyoneElse: Is there any benefit in sharing the script? If so, GitHub? Some other method?
@Uroni: Is there any issue with forcing a client restart to prevent the backup from taking place? Is there a better way of achieving the same goal of preventing the backup?
@Uroni: Following on from #2, I briefly looked through the source code but from my understanding, the prefilebackup is run but not checked for an exit code - with BackupPC, a non-zero exit code from the pre-backup script would prevent the backup from taking place - have I missed something here? Could this be something worth implementing, since it could be useful for a number of situations, if certain conditions are not in place at the time when the backup is going to run…?
@Uroni: Other than not following symlinks, is there a way of automatically recognising and avoiding the multiple symlink-trap that is found in all of the wineprefixes installed on a Linux system?
@Uroni: What do you think about using a symlink policy similar to how BackupPC operates? Symlink content vs. symlink target…
@Everyone: Not following symlinks by default for Linux: is this really, actually a better way?
Hello,
Any chance you could post the scripts you used? They would be helpful to me! Some of my Linux systems actually fill up the drive with “ERROR: Cannot stat “/usr/bin/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/X11/rvim”: Too many levels of symbolic links (40)”
and I believe your script would save me lots of time!
@Linxxs I came here to reply to you but @uroni got there first!
I’ve seen Settings > Advanced > Follow symbolic links on Linux but I haven’t got around to using it yet – good to know that this issue is now resolved, particularly looking forward to using v2.0.
thank you, I was so deep in trying to solve the problem, didn’t see that simple fix… I will try it and let you know.
Any chance 2.0 has rear integration?
I am using urbackup for windows and i have to say it works perfectly! Thank you for the software in general!
Dear Mr uroni,
I have a UrBackup Server v. 2.0.4 beta and a UrBackup Client v. 2.0.4 on a Linux machine (Ubuntu).
I can’t unfollow symlink, on the client logs I have:
Following symbolic link at “/mnt/snapshot/home/stefan/.Private” to “/home/.ecryptfs/stefan/.Private” confirms symlink backup target “.symlink_” to “/”
After having tried bacula, amanda, CrashPlan without success, I tried urbackup and I am almost satisfied.
The only problem I had was on a Linux (Ubuntu) client connecting to a urbackup server on FreeNas.
The indexing was taking forever and was filling out all my partitions - with the indexing file and with the log file.
Finally I found out that the problem was my wine installation and the symlinks loop issue.
Disabling symlinks following that was enabled by default did solve my problem, and surprise! indexing took a few seconds instead of more than 24 hours as the previous time!
As suggested by someone in the previous messages, disabling the “follow symlinks” option by DEFAULT on linux would be very useful for newbies that never faced this problem.
I am using 1.4.9 for client.
Other options are, checking for .wine directories (and other well-known apps) and alert the user for the potential problem. Or checking for loops.
Just my 0.2 cents.
Yes indeed, 2.0.x will be a solution, but if I am not wrong it is in beta status, so not for production, right?
Even if it follows the symlinks only if not already backed up, it will be stuck. Anyway this is not an issue if the user already knows of the problem
Even a timer when indexing takes too long would work. I have a big disk (2 x 1T hd, + 0.5 T SSD) with a LOT of files (compiling yocto and android in many different flavours) but indexing had taken a few seconds when working.
Is there a way to make some directory exclusion on the client side, for example, putting a special named disk (like .no_urbackup) ? it would be very useful…