Full FIle Backup

what is w.r.t.?
LOL im sorry i didn’t understand that

the urbackup server is not OMV and no docker used on the machine at all (Linux storage.techtoyoullc.com 4.19.0-17-amd64 #1 SMP Debian 4.19.194-2 (2021-06-21) x86_64 GNU/Linux)

the client has OMV (no docker either): Linux xxxxxxxx 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 GNU/Linux

So you can parhaps follow UrBackup - Install UrBackup server from sources (I’d use ./configure --prefix=/usr to replace the package version) and try if following patch helps?

diff --git a/urbackupcommon/SparseFile.cpp b/urbackupcommon/SparseFile.cpp
index 24de72af9..867c0de98 100644
--- a/urbackupcommon/SparseFile.cpp
+++ b/urbackupcommon/SparseFile.cpp
@@ -309,9 +309,10 @@ SparseFile::SPosMap SparseFile::nextBackingOffset()
 
 SparseFile::SPosMap SparseFile::lastBackingOffset(int64 offset)
 {
-	std::vector<SPosMap>::iterator it = std::lower_bound(sparse_offsets.begin(), sparse_offsets.end(), SPosMap(offset, -1));
-	if (it != sparse_offsets.end())
+	std::vector<SPosMap>::iterator it = std::upper_bound(sparse_offsets.begin(), sparse_offsets.end(), SPosMap(offset, -1));
+	if (it != sparse_offsets.begin())
 	{
+		--it;
 		return *it;
 	}
 	else

Compiling…

we’ll see what happens after I apply the patches

should I apply the patched first? or compile test and try the patch if it fails?

I lost all my config after recompiling?

any hint on where the old config with all my clients is?

There is a backup in the backup storage folder (if not disabled).

It might just use the wrong folder as db path. Check if it uses /usr/var/urbackup or /usr/local/var/urbackup as db path and create a symbolic link to /var/urbackup if appropriate.

Thank you, yes,

i just copied the database backup over to /usr/var/urbackup after i stopped the server and everything has been running since yesterday.

I’ll post the out come as soon as the backup ends

Has a backup finished and did the patch fix the issue?

It just finished and failed, without the patch:

ERROR: Client calculated hash of “/urbackup/xxxxxxx/210925-1442/dev-disk-by-label-DataA1RAID6/xxxxxxxxx_Data/fs-root/media/af0bb51b-5ab8-43f6-a444-e33cd561a431/xx_General_Files/Email Archive/xxxxxx.pst” differs from server calculated hash. This may be caused by a bug or by random bit flips on the client or server hard disk. Failing backup. (Hash: tree-sparse, client hash: C3EZO9fpH9mmLHLGwDOZ8v6eisDMWwmZPPJcJmDEAR3/M4iJttb0yx1/k9tCJenXJKTqhhq60//2a7SyIhSjMg==, server hash: WFLiRTTM18p10WUbXOpNVOk2YfPPINcRakBzxzEsZs2QRp3tFLRWQHJ180IsuH+NE1i9yoFkiLZ74RBM/mC9ow==)
WARNING: Not all folder metadata could be applied. Metadata was inconsistent.
ERROR: FATAL: Backup failed because of disk problems (see previous messages)

I’ll try patching it now and run it again

Backup restarted with this patch
(server recompiled and client downloaded from the newly compiled server)

It failed again,
Im really not sure what to do

Do you need my logs?

Could you try this patch in addition to the previous patch?

diff --git a/urbackupcommon/fileclient/FileClient.cpp b/urbackupcommon/fileclient/FileClient.cpp
index 241609606..1398adbbf 100644
--- a/urbackupcommon/fileclient/FileClient.cpp
+++ b/urbackupcommon/fileclient/FileClient.cpp
@@ -1424,6 +1424,9 @@ bool FileClient::Reconnect(void)
 						return ERR_ERROR;
 					}
 
+					if(received>0)
+						file->Seek(received);
+
 					{
 						IScopedLock lock(mutex);
 						sparse_bytes += sparse_file->getSparseSize();

I applied the patch to the server
I completely uninstalled the client, then downloaded the client from the newly compiled server and now it just started again. We’ll see what happens

Thanks for all your help

W.R.T. or WRT = with regard to

Happy to let you know that:

Level Time Message
Info 10/09/21 13:25 Starting unscheduled full file backup…
Info 10/10/21 00:07 Unmounting /dev/datto0 at /mnt/urbackup_snaps/b69d4e65d06f7f77fc1bdabacfd392f42e8d509bfc8b0fc6…
Info 10/10/21 00:07 Removing devicemapper snapshot…
Info 10/10/21 00:07 Destroying dattobd snapshot /dev/datto0…
Info 10/10/21 00:07 Snapshotting device /dev/sdb1 via dattobd…
Info 10/10/21 00:07 Using /dev/datto0…
Info 10/10/21 00:07 Mounting /dev/mapper/wsnap-a0ed2489109f61620a044ca3a4b6cc80ba6a351bf2788f98…
Info 10/10/21 00:07 Indexing of “dev-disk-by-label-DataA1RAID6” done. 57952 filesystem lookups 0 db lookups and 0 db updates
Info 10/10/21 00:07 xx-serv-w7: Loading file list…
Info 10/10/21 00:07 xx-serv-w7: Started loading files…
Info 10/10/21 12:36 Waiting for file transfers…
Info 10/10/21 18:09 Waiting for file hashing and copying threads…
Info 10/10/21 18:10 Writing new file list…
Info 10/10/21 18:10 All metadata was present
Info 10/10/21 18:10 Transferred 103.602 GB - Average speed: 13.7071 MBit/s
Info 10/10/21 18:10 (Before compression: 164.432 GB ratio: 1.58715)
Info 10/10/21 18:10 1.0503 TB of files were already present on the server and did not need to be transferred
Info 10/10/21 18:10 Time taken for backing up client xx-serv-w7: 1 days 4h 45m 29s
Info 10/10/21 18:10 Backup succeeded
2 Likes

Sorry to bring this back to life again.

What do I do to upgrade to the latest server/client version and keep this patch?

Was this issue fixed in future versions?

Im asking bc now the client beckups are not starting and the debug logs dont show much except:

2022-05-25 08:08:27: msg=START BACKUP INCR
2022-05-25 08:08:27: Cannot do image backup because can_backup_images=false

that is irrelevant bc im trying to do incremental file backup. Correct?