Taking ages with full backup - even though it says 100%?

Andy_Newby · September 21, 2021, 2:09pm

Hi,

I’m trying to work out why a full backup (files, not image) is taking so long. Its going from Ubuntu to Ubuntu (client and server)

Here is what I see:

Then when I look at Activities, I see:

When looking at “View Log”, I see just a load of these events:

|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960701-5.jpg (id=591412)|
|---|---|---|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/496071-DuZGKi4hgxKppPZPBTkhYQ.jpg (id=591413)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960711-2.jpg (id=591414)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960721-12.jpg (id=591415)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960731-9.jpg (id=591416)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960741-8.jpg (id=591417)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960751-1.jpg (id=591418)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960761-6.jpg (id=591419)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960791-10.jpg (id=591420)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960801-7.jpg (id=591421)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/496081-1axZmL5OSi4jeoXPtXAQwA.jpg (id=591422)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960811-7.jpg (id=591423)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960821-7.jpg (id=591424)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960841-4.jpg (id=591425)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960861-7.jpg (id=591427)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960871-5.jpg (id=591428)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960891-2.jpg (id=591429)|
|21/09/21 15:07  |DEBUG  |HT: Linked file: /BACKUP/urbackup/admin.xxxxx.org/210921-1141/home/chambres/web/xxxxx.org/public_html/links/uploads/original/1/4960901-4.jpg (id=591430)|

Why does it tell me its at 100%, when its still taking a good couple of hours after doing all this stuff?

The RAM is using up 3gb out of 4gb - but the CPU (dual processor) is only hitting 10-20% on each one… so doesn’t seem like its maxed out? We upgraded this server from a single to dual-processor (and double the RAM) to try and speed it up, but it still seems quite slow (incremental backups are decent, but full ones seem to take many many hours)

Very confusing

BTW: Does all that “DEBUG” logging slow the process down? I couldn’t seem to find anywhere that would lower the verbosity of it?

Thanks in advance!

Andy

MrBates · October 6, 2023, 8:31am

Same thing happened to me. It seems to be updating the internal database with the files info, but for me it took over 24 hours after a full file backup was supposedly finished… During this time no other backups (even image backup) could be performed from this client.

anon16079863 · October 6, 2023, 11:48am

Just a guess. Your CLIENT is 100% done, but the SERVER is still calculating hashes and stuff.

You are running a FULL file every time correct? To me that means every single file has to be rehashed every single backup.

The only time I have bumped into something similar happening during incremental backups, is when I had one client connected to 2 servers. For some reason it rehashed every single file even if it was an incremental.

Andy_Newby · October 6, 2023, 1:30pm

Hi,

I managed to get around it by adding a lot of extra “excludes”. I guess it didn’t like the number of small files. Its now ticking over fine (it would be nice to have seen a bit more information what it was backing up so I could see what was happening; and if it was backing up lots of files I didn’t want :))

Cheers

Andy

anon16079863 · October 6, 2023, 1:36pm

“Files you don’t want”… Just open the backup in the web gui and LOOK what files are backed up.
If you want even more info, change the log level and read the logs (warning, they can become VERY long) or just watch the live log run (by pressing the button on the web gui activities page when the backup is running) and study what it is doing.

Andy_Newby · October 6, 2023, 1:54pm

There was 100gb worth of files - so can’t really watch it in the log unfortunately (you would loose a day otherwise). The problem with the GUI, is that you have to drill down and look - there are 100+ websites, all with their own setups, so its not like ncdu where you can just drill down and look for large folders, if you see what I mean

Anyway - its all good for now . Thanks

MrBates · October 6, 2023, 3:10pm

I ran a full file backup because I wanted to start fresh after a database corruption. Usually I’m doing only incrementals.

The indexing/hashing/whatever it is doing is going to take another 24 hours according to my calculations (the speed went down from 2 files per second to 1 file), which would make it 48 hours in total.

I need to physically move the server to another location, but I don’t know if shutting it down would damage this operation or whether it would resume it afterwards. Any idea?

MrBates · October 7, 2023, 4:11am

For whoever is interested, it was a bad idea to stop it at this stage. After restarting, the full backup started from scratch.