Slow backup processing, move instead of copy files?

Hi, I have some big files to backup, zfs backend internet only mode 2.4.13 server version.
It transfers everything with acceptable speed, writes files in /urbackup/files/xxx/210421-1133/.b68jlqwehf folder.
Which is on the same zfs, than final. After that, the backup starts to copy files from /urbackup/files/xxx/210421-1133/.b68jlqwehf to the final place, witch takes lot of time, nearly the same as the backup, instead of moving the file to the new location.
I misconfigured something, or it is the normal behaviour?

afaik urbackup is not only copying files from temporary to final destination. It also doing checksumming on the files. This is done in a single thread afaik. This point could be enhanced.
Check this out:
Also check your backup settings: prefer “block differences - hashed” if your big files have only small changes.

Hi Knuuut!
Thank you for the answer.
I have alredy read that part of documentation, the temporary buffers are off(new behaviour only), and the transfer is set to raw.
Basically I do not want to make any change on theese settings, to save cpu on clients. And the speed is really ok, 6 TB 8-16 hours. After that no network traffic. But processing 2 of 6 0.8 tb files took 24 hours, and it looked like slow copy on the same filesystem. Another 4 big files, and 3000 small files moved immediatelly.
Afaik, with zfs the deduplication is done by the the filesystem itself.
And looks like not all files are copyed that way, some of them are moved.
I am still searching the reason, but I hoped somebady has similar experience.

Could you past an excerpt from a server debug log that shows the files being copied? Thanks!

Settings like the “raw” setting, internet/local client, client side hash calc etc. would be useful as well.

The internet backups all set to raw, local backups all set to hashed. This particular backup is local backup, for internet backups encrypted transfer, and Calculate file-hashes on the client: are on.
Maybe this important, Full backups are disabled, this is second backup of a database backup, which is owerwritten weekly. So all files are replacet weekly base, and during the week just few small new files added.

The debug log rotated out! :frowning: This backup runs weekly, this week was ok, took 13 hours. Last week took 2 days an 9 hours. the size was 3.8 Tb this time, in slow scenario was 4.3 TB.

I am started whatching for it, and if it happens again, I vill attach logs, and lsof logs. and screen caps.


Hi Uroni!

The backup is not finished yet, filetransfer in process, but lsof

urbackups 3006480 root 109r REG 0,822 864447275008 366146 /backups/urbackup/files/xxx/210503-0001/.b68xO+K9SCOF35cLk4Bf9Q/23

this is the destination file:
urbackups 3006480 root 424w REG 0,822 620369260544 366348 /backups/urbackup/files/xxx/210503-0001/backup/p1.dbf

another file is under creation in tmp area
urbackups 3006480 root 307u REG 0,822 525957858728 363854 /backups/urbackup/files/xxx/210503-0001/.b68xO+K9SCOF35cLk4Bf9Q/26
urbackups 3006480 root 424w REG 0,822 620376465408 366348
I can’t see anything in urbackup.log yet.
Possible it starts copying file before the transfer finished?