Before I start on anything new, I’d like your opinion on some larger features. I don’t want to build something that doesn’t fit your plans for UrBackup, and I know there’s already a queue of my PRs waiting for review, so there’s no rush.
Monitoring. A Prometheus-style metrics endpoint (last success, backup age, size, errors per client). People have been asking for this since 2016 and have been using external exporters. I’d also add a stock “webhook” alert script next to the Pulseway one, sending JSON for ntfy/Slack/Discord etc.
Automatic restore testing. On a schedule, the server mounts a random recent image or file backup, checks it and compares the hashes of a sample of files. Each client then shows a “last verified restorable” date.
Off-site copy. Finished backups get copied to a second server or to S3 storage with Object Lock (immutable for N days). This gets asked a lot, but I know replication is an appliance feature, so I’d rather ask first: is something like this OK in the open-source server, or would it conflict with the appliance?
Protecting old backups during an attack. Not detection (I read your reasoning in the old ransomware thread, and the canary covers that), only this: if the canary fails, or an incremental changes far more than usual, pause automatic cleanup for that client so older good backups aren’t deleted before someone looks.
Container and database awareness. Optionally dump or quiesce databases in Docker/Podman containers before a file backup, controlled by container labels.
Search for a file across all backups and clients (“every version of budget.xlsx”).
(The TOTP 2FA question is in the “2FA login to Web UI possible?” thread.)
My plan would be to start with 1 because it’s small and self-contained, then 2.
Questions:
Are any of these things you’re already working on, or ones you’d rather not have in the main code?
Is there a design direction you’d prefer for 3 and 4?
Should new features target dev only, or also 2.6.x?
The problem is that this cannot be done with 100% reliability. I know that commercial competitors do this, so it might be needed but I question the value – the system just needs to function reliably.
The appliance heavily leans on btrfs features for this, which won’t be possible for UrBackup itself. Implementing this with acceptable performance would be hard.
A way to go would be to just call something like restic for this? I guess it would be easier for users if this is directly integrated.
Yeah, I think I also need to find canary code and finally put that in. If the canary fails, it fails backups, so old backups shouldn’t cycle out anyway.
Sounds good.
Appliance also implements this but uses btrfs features for this.
I’ll start with monitoring (the metrics endpoint plus a webhook alert script), then do the container/database labels. I’ll drop 2, 4 and 6. For off-site copies, restic integration is a good idea. I’ll come back to it once the other two are done.
One question: should these go to dev only, or also to 2.6.x?