Urbackupsrv core dump during repari-database

GM,

  1. some of my clients seem unable to complete incremental or full file backups (but images backups are working fine)

  2. Assuming that the issue above was related to a corrupt DB, I have tried to run repair_database but it cause the process to core dump.

Running latest version of urbackupsrv on ubuntu. Clients are running latest version of urbackupclient on windows 11 pro. client with the issue is backing up over Lan.

Here the output of running repair-database:

root@hal:/usr/bin# ./urbackupsrv repair-database  -u urbackup
2024-05-11 08:23:27: SQLite: recovered 8720 frames from WAL file /var/urbackup/backup_server.db-wal code: 283
2024-05-11 08:23:27: SQLite: recovered 69408 frames from WAL file /var/urbackup/backup_server_files.db-wal code: 283
2024-05-11 08:23:27: SQLite: recovered 8 frames from WAL file /var/urbackup/backup_server_link_journal.db-wal code: 283
2024-05-11 08:23:27: SQLite: recovered 7 frames from WAL file /var/urbackup/backup_server_settings.db-wal code: 283
2024-05-11 08:23:27: SQLite: recovered 8720 frames from WAL file /var/urbackup/backup_server.db-wal code: 283
2024-05-11 08:23:27: Recovering contents of database with id 20...
Segmentation fault (core dumped)

Here is the output of dmesg matchingt he core dumps:

[435775.009718] urbackupsrv[2594475]: segfault at 8 ip 000061b0927c8bb1 sp 00007fff3918f550 error 4 in urbackupsrv[61b0926f6000+6de000] likely on CPU 0 (core 0, socket 0)
[435775.009737] Code: c5 02 00 66 0f 1f 44 00 00 48 89 da 48 8d 35 d6 f5 fe ff e8 11 4b 02 00 eb b8 0f 1f 80 00 00 00 00 48 8b 83 c0 11 00 00 89 f5 <4c> 8b 60 08 0f b6 43 0d 84 c0 0f 85 a7 03 00 00 4d 85 e4 74 0b 41
[435963.775823] urbackupsrv[2595645]: segfault at 8 ip 0000623101c98bb1 sp 00007ffff4d51900 error 4 in urbackupsrv[623101bc6000+6de000] likely on CPU 2 (core 2, socket 0)
[435963.775855] Code: c5 02 00 66 0f 1f 44 00 00 48 89 da 48 8d 35 d6 f5 fe ff e8 11 4b 02 00 eb b8 0f 1f 80 00 00 00 00 48 8b 83 c0 11 00 00 89 f5 <4c> 8b 60 08 0f b6 43 0d 84 c0 0f 85 a7 03 00 00 4d 85 e4 74 0b 41
[437267.986114] urbackupsrv[2603462]: segfault at 8 ip 00006141b108bbb1 sp 00007ffcd0ebbe00 error 4 in urbackupsrv[6141b0fb9000+6de000] likely on CPU 1 (core 1, socket 0)
[437267.986141] Code: c5 02 00 66 0f 1f 44 00 00 48 89 da 48 8d 35 d6 f5 fe ff e8 11 4b 02 00 eb b8 0f 1f 80 00 00 00 00 48 8b 83 c0 11 00 00 89 f5 <4c> 8b 60 08 0f b6 43 0d 84 c0 0f 85 a7 03 00 00 4d 85 e4 74 0b 41
[437272.550854] urbackupsrv[2603495]: segfault at 8 ip 000062612bb1abb1 sp 00007ffcbdb000e0 error 4 in urbackupsrv[62612ba48000+6de000] likely on CPU 0 (core 0, socket 0)
[437272.550885] Code: c5 02 00 66 0f 1f 44 00 00 48 89 da 48 8d 35 d6 f5 fe ff e8 11 4b 02 00 eb b8 0f 1f 80 00 00 00 00 48 8b 83 c0 11 00 00 89 f5 <4c> 8b 60 08 0f b6 43 0d 84 c0 0f 85 a7 03 00 00 4d 85 e4 74 0b 41
[515672.058189] urbackupsrv[3069114]: segfault at 8 ip 0000560a5abf7bb1 sp 00007ffc8567a670 error 4 in urbackupsrv[560a5ab25000+6de000] likely on CPU 2 (core 2, socket 0)
[515672.058220] Code: c5 02 00 66 0f 1f 44 00 00 48 89 da 48 8d 35 d6 f5 fe ff e8 11 4b 02 00 eb b8 0f 1f 80 00 00 00 00 48 8b 83 c0 11 00 00 89 f5 <4c> 8b 60 08 0f b6 43 0d 84 c0 0f 85 a7 03 00 00 4d 85 e4 74 0b 41
[515692.933099] traps: InternetService[3031826] general protection fault ip:5759cc933213 sp:73eafefec9c0 error:0 in urbackupsrv[5759cc5a6000+6de000]
[515698.722137] urbackupsrv[3069289]: segfault at 8 ip 000059f80a287bb1 sp 00007ffc65f1a240 error 4 in urbackupsrv[59f80a1b5000+6de000] likely on CPU 0 (core 0, socket 0)
[515698.722159] Code: c5 02 00 66 0f 1f 44 00 00 48 89 da 48 8d 35 d6 f5 fe ff e8 11 4b 02 00 eb b8 0f 1f 80 00 00 00 00 48 8b 83 c0 11 00 00 89 f5 <4c> 8b 60 08 0f b6 43 0d 84 c0 0f 85 a7 03 00 00 4d 85 e4 74 0b 41
[515705.726420] urbackupsrv[3069334]: segfault at 8 ip 00005863e4a0cbb1 sp 00007ffca25f2340 error 4 in urbackupsrv[5863e493a000+6de000] likely on CPU 0 (core 0, socket 0)
[515705.726444] Code: c5 02 00 66 0f 1f 44 00 00 48 89 da 48 8d 35 d6 f5 fe ff e8 11 4b 02 00 eb b8 0f 1f 80 00 00 00 00 48 8b 83 c0 11 00 00 89 f5 <4c> 8b 60 08 0f b6 43 0d 84 c0 0f 85 a7 03 00 00 4d 85 e4 74 0b 41
[

Same thing happening in the latest docker container

Please follow instructions at Recovering Data From A Corrupt SQLite Database for now. But preferably restore the damaged database from backups. Repair might break it even more.

It is best to think of the recovery API as a salvage undertaking. Recovery will extract as much usable data as it can from the wreck of the old database, but some parts may be damaged beyond repair and some rework and testing should be performed prior to returning the recovered database to service.

ouch I see…
in that case, is there a simple way to simply wipe out the db completely and let clients start over? in this case I don’t need to get back older files, I am using urbackup in case of hd/ssd failures only.
Ideally preserving the settings though…