WARNING: SQLite: database corruption at line 56972 of [3bfa9cc97d] errorcode: 11

I’ve just tried to follow the instructions from https://urbackup.atlassian.net/wiki/spaces/US/pages/4292612/Troubleshooting+UrBackup+Server+Database and found out that VACUUM command errors with “database or disk is full”. Indeed, sqlite3 uses /var/tmp as a temp folder and it does not have enough free space.
I was able to change the temp folder in sqlite by using PRAGMA temp_store_directory=.
I guess, urbackup might face the same issue. Just set SQLITE_TMPDIR var in /etc/sysconfig/urbackup-server and no more corruption warnings! still see the warning :frowning:

It’s disappointing to find out that /var/tmp is being used, since on my setup it is not encrypted. This is why I mounted /var/urbackup to the encrypted storage. Thanks for sharing this.

Me too, and the corruption warnings still show.

In Ubuntu it’s in /etc/default/urbackupsrv, but I can’t confirm the setting works because the newly set directory is empty after starting the service.

I believe you can check the free space (using df) before and during the repair-database command running. If you see the free space is less than before, then that folder is in use

Does anybody found solution for this error? Got it on URBackup running on Windows in Hyper-V.

As I understand it, all administrators of Urbackup instances are eventually confronted with this error: the database is corrupted at line 56972. :face_with_monocle:

For my part, I discovered this error when another error had just occurred on my current Urbackup instance (hosted on Proxmox Backup Server). But this same error was already present on my previous instance, hosted on Ubuntu Server, and had been for several years if you consider that this error occurs with every update…

However, this error didn’t prevent the application from working perfectly well, so it’s still a consoling piece of information. Just don’t launch the correction tool, and you’ll be left in the dark. :face_with_thermometer:

But what does this phase mean: the database is corrupt at line 56972: which table is affected and in which database?

We don’t know… It’s a shame. Could the error message be improved? :sunglasses:

Thanks in advance to the developers for looking into this bug, which appears to be structural and seems to be present on many instances. :hugs:

Ernest.

I’ll also throw my hat into the ring on this as I’m also seeing this error.

# urbackupsrv repair-database -u root
2024-01-10 22:19:55: Recovering contents of database with id 20...
2024-01-10 22:19:55: Recovering contents of database with id 30...
2024-01-10 22:19:55: Recovering contents of database with id 23...
2024-01-10 22:19:57: WARNING: SQLite: database corruption at line 56972 of [3bfa9cc97d] errorcode: 11
2024-01-10 22:20:15: WARNING: SQLite: database corruption at line 56972 of [3bfa9cc97d] errorcode: 11
2024-01-10 22:20:23: WARNING: SQLite: database corruption at line 56972 of [3bfa9cc97d] errorcode: 11
2024-01-10 22:21:39: WARNING: SQLite: database corruption at line 56972 of [3bfa9cc97d] errorcode: 11
2024-01-10 22:21:39: Recovering contents of database with id 24...
2024-01-10 22:21:47: Recovering contents of database with id 25...
2024-01-10 22:21:47: Importing database with id 20...
2024-01-10 22:21:47: Importing database with id 30...
2024-01-10 22:21:47: Importing database with id 23...
2024-01-10 22:23:21: Moving rows from lost+found...
2024-01-10 22:23:21: Importing database with id 24...
2024-01-10 22:23:32: Importing database with id 25...

I’m currently using UrBackup Server 2.5.32 on a Rocky Linux 8.9 LXC container on Proxmox 8.1.3. The datastore for this server is on ZFS and the server has ECC memory, so I have a hard time believing that it’s a corruption issue.

The only reason I ran into this issue is related to troubleshooting a different issue about a client using too much space.

Given the diverse hardware/systems in this thread, and the fact we are all receiving the exact same error, it seems like it shouldn’t be due to corruption, however I’m not exactly sure what that warning message means, so I could be misunderstanding it.

Looking at the code, the function that actually generates the error message in the logs appears to be errorLogCallback, which is registered with sqlite3_config here. It appears that database corruption at line 56972 of [3bfa9cc97d] is coming from SQLite itself.

A cursory Google for “database corruption at line” came up with very little. Perhaps there’s a nugget buried somewhere that can be revealed with better search terms or a direct dive into SQLite’s code.

EDIT: I found (I think) the relevant functions in SQLite’s code. sqlite3CorruptError makes the call to sqlite3ReportError, which is the function that actually prints the output. I’m unable to deduce what lineno is actually referring to, or how I’d go about pulling that data out of the UrBackup SQLite db in order to investigate further.

It is line 56972 of this file https://github.com/uroni/urbackup_backend/blob/2.5.20/sqlite/sqlite3.c

Which is

/* The pager cache has created a new page. Its content needs to 
    ** be initialized. But first some error checks:
    **
    ** (1) The maximum page number is 2^31
    ** (2) Never try to fetch the locking page
    */
    if( pgno>PAGER_MAX_PGNO || pgno==PAGER_MJ_PGNO(pPager) ){
      rc = SQLITE_CORRUPT_BKPT;
      goto pager_acquire_err;
    }

I assume it’s caused by a previous out of memory on temp storage.