Hello everyone, we have been testing UrBackup in our companies environment.
We work as an MSP and our idea was to use this to do workstation backups for workstations of our clients. The problem we have been facing is with SQLite and the fact that database will become way to big as we would have around 500 workstations and are aware that SQLite has a tendency to become unstable when it reaches a certain threshold. We would like to request a feature to use mysql or postgresql as a backend database, as a feature that we would be willing to pay for
I’m not aware of any such issue. Please describe symtoms, steps to reproduce, software versions involved, logs etc. Thanks!
I have been having issues with SQLite databases in some servers sporadically. The issues starts with the message in the web interface that says that UrBackup cannot read database.
After this, I always run repair-database command and returns things like this:
2020-02-14 10:57:41: SQLite: recovered 14 frames from WAL file /usr/local/var/urbackup/backup_server_settings.db-wal code: 283
2020-02-14 10:57:41: Exporting database with id 30…
2020-02-14 10:57:49: SQLite: recovered 190063 frames from WAL file /usr/local/var/urbackup/backup_server_files.db-wal code: 283
2020-02-14 10:57:49: Exporting database with id 23…
2020-02-14 10:59:49: WARNING: SQLite: database corruption at line 64011 of [0ee482a1e0] errorcode: 11
2020-02-14 10:59:49: WARNING: SQLite: statement aborts at 2: [SELECT ‘INSERT INTO ’ || ‘“files_incoming_stat”’ || ’ VALUES(’ || quote(“id”) || ‘,’ || quote(“filesize”) || ‘,’ || quote(“clientid”) || ‘,’ || quote(“backupid”) || ‘,’ || quote("existi errorcode: 11
2020-02-14 10:59:49: WARNING: SQLite: database corruption at line 64011 of [0ee482a1e0] errorcode: 11
2020-02-14 10:59:49: WARNING: SQLite: statement aborts at 3: [SELECT ‘INSERT INTO ’ || ‘“files_incoming_stat”’ || ’ VALUES(’ || quote(“id”) || ‘,’ || quote(“filesize”) || ‘,’ || quote(“clientid”) || ‘,’ || quote(“backupid”) || ‘,’ || quote("existi errorcode: 11
2020-02-14 10:59:49: Exporting database with id 24…
2020-02-14 10:59:49: SQLite: recovered 1 frames from WAL file /usr/local/var/urbackup/backup_server_link_journal.db-wal code: 283
All my servers are running in Linux Debian 8 or 9 with Btrfs in root partition and data partition, some of them are Virtual machines, some are phisical servers, some are NAS (i.e. Netgear wich have an embeded Debian 8). I have experienced this class of problems in all of them.
This is the major issue I am having currently with UrBackup, I need to stay very attent to ensure that all servers are working correctly every day.
Many thanks.
I guess this is with the newest UrBackup version? If you are using btrfs I would recommend using Linux kernel 4.14.y or 4.19.y. Also have a look at the dmesg output. Perhaps sectors of the storage become unreadable…
It is unlikely that this is a SQLite problem. SQLite is used in e.g. planes (Airbus)…
thanks @uroni i will test it out by making a larger dataset and see if i start seeing problems. I do have one question though, i have not been able to find while going through the docs, what exactly does urbackup store in the database? e.g configurations for the server/clients, logs, statuses, it would be really great if i could get some clarification, as unfortunately im still a bit of a novice when it comes to DB administration so going through them would be a bit of a problem. Thank you
It stores this kind of stuff as well as file timestamps, but mainly hash-> file path, makes the bulk of the databases size.
I have like 250 client and i am not encountering corruption issues.
Maybe you could try running docker and have more than one urbackup instance each serving a different subnet ?