Can't backup since 1803 update of Windows 10

[menzo]

Since I’ve installed the 1803 update of Windows 10 on 2 client machines, indexing for backups fails.

I’m using client version: 2.2.5
and server version: 2.2.11

This is the relevant log the server provides:

2018-05-02 17:08:01: ERROR: Cannot access path to backup: “\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy3\Users\Liz Leenders\OneDrive” Errorcode: 19 - Het medium is tegen schrijven beveiligd.
2018-05-02 17:08:01: ERROR: Constructing of filelist of “Laptop-Liz” failed: error - index error
2018-05-02 17:08:04: ERROR: Backup had an early error. Deleting partial backup.
2018-05-02 17:08:05: WARNING: Exponential backoff: Waiting at least 2h 40m before next file backup

It says the media is write protected. These clients worked fine minutes before the update.

[slikvik]

Exact same issue here. I suspect it’s to do with the fact that OneDrive has a new feature where it only downloads the file if you access it otherwise it acts as a filesystem “stub” of sorts.

[mds]

me too!

[rhetor]

This is a problem with VSS. Hopefully MS resolves the issue soon. In the meantime try disabling the feature in OneDrive that downloads on demand, “Files On-Demand”

[Greg]

Also affected by this issue.

I’ve tried excluding the OneDrive folder, but does not seem to help:

Excluded files (with wildcards):

*\System Volume Information\*;*\$WINDOWS.~BT\*;*\DfsrPrivate\*;*\AppData\Local\Packages\Microsoft.MicrosoftOfficeHub_8wekyb3d8bbwe\LocalCache\*;*\OneDrive\*

[silversword]

Used this to find out if the “Files On-Demand” is enabled for the PC.

For UrBackup compatibility I unchecked this

OneDrive Files On-Demand: How to enable it in Windows 10 Fall Creators Update

[ravensorb]

I am curious, why would this impact files that are actually downloaded. Shouldn’t urbackup be able to handle those? Maybe fall back to a standard file backup?

[silversword]

@ravensorb

Shouldn’t urbackup be able to handle those?

My Hypothesis:
I think Microsoft is using a custom file hook into the file system to “simulate” a file showing up even though it may, or may not be in the file system and they haven’t done VSS compatibility testing on their shim.

VSS getting access errors as a result. UrBackup uses the native VSS subsystem for snapshot/file access to backup

[uroni]

I haven’t tested it, but maybe it is able to read them if you run the service as a different user (the FAQ shows how to do this https://www.urbackup.org/faq.html#use_shares ).

This would then download all files, so you might as well just turn “Files On-Demand” off.

Or you can exclude it and make sure the OneDrive cache folder is backed up. Wherever that folder is.

In my opinion it seems that if you use OneDrive in this mode, backups are then responsibility of Microsoft. Have a look at their SLAs, and if they are not reliable enough for your needs, don’t use it.

[hexadecagram]

I have danced around this issue by adding *\OneDrive* to my Excluded files list and also have disabled the “Files On-Demand” feature on several PCs in my network Now, the contents of Documents, Pictures, Videos, and many other folders (Including %APPDATA% for each user) are not being backed up. Instead, the head of each folder is backed up as a 26 byte file, which turns out to be 0 bytes when downloaded.

In short, those folders are not being descended or backed up at all.

I would regard this as a major issue because it effectively defeats the purpose of urbackup apart from image backups.

[orogor]

Hi

My understanding is that the files may actually not be on the drive.
When you access the one drive folder, you’re effectively accessing a Microsoft share.

Then one drive keep some kind of local cache of the files you accessed and there’s some sync going on.
But these are not files in the traditional way, even if windows try to abstract this very well at present these as files in explorer.

A test to do if you have access to another onedrive.
Look at the disk usage of c: or whatever.
Copy a >1GB file from another onedrive to yours using the web ui.
Just after, check if you see the file locally in the onedrive folder. Check the disk usage as well.

My understanding is that you’ll see the file in your local onedrive copy.
But for sometime the real disk usage will stay unchanged.
Thus the file would not be actually present.
Thus urbackup wouldn’t be able to backup it.

[silversword]

Checking here folders for OneDrive are backing up fine.

Note. “Files On-Demand” is unchecked in the OneDrive preferences. There is no exclusion for OneDrive in UrBackup settings. Auto Save is not enabled (aka make symlinks of your desktop/documents/pictures folder and map it to the Onedrive folder)

OneDrive Files are replicated to the c:\Users\%UserName%\OneDrive\ folder
Those files are in UrBackup File backup.

@hexadecagram

adding \OneDrive to my Excluded files list

note the forums sometimes strips out characters if you don’t frame paths as preformatted text.
If a *\OneDrive\* exclusion exists you are telling it to NOT backup OneDrive anywhere, including the default location.

@orogor

Look at the disk usage of c: or whatever.

I did this test and disk usage changed. Note: The reason was because the “Files On-Demand” feature is unchecked. Of note: Using the Glasswire firewall, I also see the data transfer of Onedrive downloading the 1GB file as double-confirmation.

[Flatlinebb]

I am having the same issue as described above, but all my attempts to exclude the OneDrive folder from backups have been ignored. Here is my entry for Exclude Files for the Windows 10 client:

*\Temp\*;C:\$WINDOWS.~BT\*; \\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy*\*;*\WINDOWS\*;*\HarddiskVolumeShadowCopy\*;*\YCOXGDawKJFxIyK2LFp4\*;*\ShadowCopy\*;*\SafeOS\*;*\AppData\Local\*;*\Definition Updates\*;*\cache2\*;*\Microsoft Antimalware\*;C:\ProgramData\*;*\Dropbox\*;*\AppData\*;*\OneDrive\*;*.iso;*\OneDrive\*;*/Users/:/OneDrive/*;*\Users\:\OneDrive\*

As you can see, I'm even attempting to use the Linux slashes in addition to the Windows backslashes, just in case. But the OneDrive folder does not get skipped, as the error log shows:

|08/02/18 15:01  |INFO  |Waiting for file transfers...|
|08/02/18 15:01  |ERROR  |Error getting complete file "YCOXGDawKJFxIyK2LFp4|Users/xxxxxxx/OneDrive" from Pxxxxxxx58. Errorcode: CANNOT_OPEN_FILE (3)|

I have restarted the client service and the server service, even though I don’t think I should have to. I have not tried disabling FIles On Demand feature in OneDrive, as it doesn’t seem to apply if I’m just trying to skip backing up OneDrive files altogether.

EDIT:
So I tested by disabling Files on Demand in OneDrive:

And the backup successfully skipped the OneDrive folder and there were no errors in the log. Yay! I reenabled the Files on Demand checkbox, ran another incremental file backup and Boo! Same error in the log.

So going forward, I will know to uncheck that, even though it seems very odd that one checkbox would make a difference. I guess MS is doing some magic behind the scenes that affect how urbackup sees files on the drive.

[silversword]

@Flatlinebb

And the backup successfully skipped the OneDrive folder and there were no errors in the log.

…I swear that’s what I said a while ago.

[uroni]

If you want to exclude it you need to exclude the folder not the folder contents (with e.g. adding C:\Users\:\OneDrive to the excludes).

Also, I have tested this, unless one adds the OneDrive folder as backup dir the backup succeeds with an ERROR message in the logs?

[silversword]

@uroni

If you want to exclude it you need to exclude the folder not the folder contents (with e.g. adding C:\Users\:\OneDrive to the excludes).

Checking: UrBackup - Server administration manual
and http://urbackup_server_name:55414/help.htm#exclude_files

I don’t see the notation format of what the : means in the exclude context. Is this listed somewhere else?

Can you point me to where those two items in the source code are? I think one of those is:

uroni/urbackup_backend/blob/2.2.x/urbackupserver/doc/admin_guide.tex

which is in a special format. Is there an editor/process to assist with the formatting of this, would like to help enhance if possible.

[XiteHosting]

Hello,
I’m having the same issues. But my problem is I DO want to back-up the OneDrive folder.

This has always worked. Even with the files-on-demand feature, although, the feature had to be disabled.
But after disabling the feature the back-ups ran fine.

Since 1803 indeed the back-ups don’t seem to work, even after turning of the files-on-demand feature.
I have now already a couple of clients with this issue.

Are there things I can do / try t to help you debug this issue?

Disabling the “Volume Shadow Copy” service does seem to work at the moment, but this is not something I would like to keep

thanks,
Stijn

[krsz]

@XiteHosting

Disabling the “Volume Shadow Copy” service does seem to work at the moment, but this is not something I would like to keep

Can you tell me how to do it? Disabling VSS in windows services prevents backup execution

backupcom->InitializeForBackup() failed. VSS error code VSS_E_UNEXPECTED

[uroni]

@XiteHosting

Since 1803 indeed the back-ups don’t seem to work, even after turning of the files-on-demand feature.
I have now already a couple of clients with this issue.

Are there things I can do / try t to help you debug this issue?

Disabling the “Volume Shadow Copy” service does seem to work at the moment, but this is not something I would like to keep

Could you tell us the errors in case files-on-demand is enabled?

Also interesting would be, what kind of reparse point OneDrive is using with e.g.
fsutil reparsePoint query C:\Users\USER\OneDrive

[XiteHosting]

Hello,
Sorry for the long delay. But it seems this has popped up again. (Probably after the update to 1809).
I haven’t had the time to connect to a client system to try the fsutil query.

The error message I receive is this:
UrBackup just did a full file backup of “xxx Laptop”.

Report:
( 9 infos, 0 warnings, 4 errors)

2019-01-21 21:28:32(info): Starting scheduled full file backup…
2019-01-21 21:33:04(info): Indexing of “Pictures” done. 5 filesystem lookups 0 db lookups and 0 db updates
2019-01-21 21:33:04(info): Indexing of “Desktop” done. 2 filesystem lookups 0 db lookups and 0 db updates
2019-01-21 21:33:04(info): Indexing of “Contacts” done. 1 filesystem lookups 0 db lookups and 0 db updates
2019-01-21 21:33:04(info): Indexing of “Documents” done. 249 filesystem lookups 0 db lookups and 0 db updates
2019-01-21 21:33:04(info): Indexing of “Dropbox” done. 59 filesystem lookups 0 db lookups and 0 db updates
2019-01-21 21:33:04(info): Indexing of “Favorites” done. 2 filesystem lookups 0 db lookups and 0 db updates
2019-01-21 21:33:04(info): Indexing of “Links” done. 1 filesystem lookups 0 db lookups and 0 db updates
2019-01-21 21:33:04(info): Indexing of “Music” done. 1 filesystem lookups 0 db lookups and 0 db updates
2019-01-21 21:33:04(error): Cannot access path to backup: “\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy1\Users\xxx\OneDrive” Errorcode: 19 - Het medium is tegen schrijven beveiligd.
2019-01-21 21:33:04(error): Indexing files failed, because of error
2019-01-21 21:33:04(error): Constructing of filelist of “xxx Laptop” failed: error - index error
2019-01-21 21:33:13(error): Backup had an early error. Deleting partial backup.

On the previous version of windows my work-around was to disable the VSS service.
I will try to take over a client’s pc and test the fsuitl command, and then try if disabling VSS solves it for now.

Is there something else I can try?

regards,
Stijn