Hello everyone. I have a strange issue that seems to be getting worse.
I have several UrBackups servers running with anywhere from 5 to 25 clients per server connected. These clients only make image backups (for full restore options) and this has been working perfectly for most of the clients connected.
However i’m noticing more and more frequently that there are clients that are not getting backupped. Meaning 14 out of 15 clients will make incremental and full backups on schedule as they should. But one client is not being backed-up. I can manually start a incremental or full image backups and it will make a backup as normal. But then a few days later I notice that again, that client has not been backupped automatically.
The strange thing is dat there is plenty of storage, the backup window is 24x7 , there is no client specifiek setting for the backups, there’s nothing in the logs about errors when making a backup. In fact, there are no logs of the missing days when backups should have been triggered automatically. Manual backups work perfectly. It is specific to certain clients (as in, it’s not that the clients missing backups change from client to client). Some of these clients are locally connected, others through internet.
As far as I can see, the UrBackup Server just refuses to backup certain clients automatically, or simply ignores them. Now i’m sure I can create a python script that’ll access the API, find any servers that have missed their backup schedules and start a new backup that way, but i’d rather not create a bandaid but fix the problem.
Have any of you had this issue before?
1 Like
Just as a update for anyone finding this topic at a later date. I think it’s related to the exponential backoff, but I can’t be sure. The backups just arent triggered by the server. I resolved this by creating a python script to access the server through the api, check if any image backups have not run as they should have been according to their schedule + 4 hours, and then trigger a incremental or full backup, based on if the next backup is supposed to be a incremental or full backup (again, through accessing the settings). It also won’t run the backup again if it’s already running for the client and it wont start a backup if there are already 2 backups running…
This python script then runs every hour so it should solve my problems for now, whatever’s causing them…
#!/usr/bin/python
import urbackup_api
import time
import datetime
server = urbackup_api.urbackup_server("http://127.0.0.1:55414/x", "admin", "yourpasswordhere")
clients = server.get_status()
numfailed = 0
numtasks = 0
lastbackup = ""
#Logging to log file, specify path below if you want a different one.
log_file = open("/var/log/urbackup-manualsched", "a")
runningtasks = server.get_actions()
settings = server.get_global_settings()
max_fullimage_age = settings["update_freq_image_full"]
max_increimage_age = settings["update_freq_image_incr"]
max_simbackups = settings["max_sim_backups"]
diff_time = int(settings["update_freq_image_incr"]) + 14400 #max time for incremental image backup + 4 hours in seconds
#Start logging
curdate = datetime.datetime.fromtimestamp(time.time()).strftime("%x %X")
print("---------------------------------------------------", file=log_file )
print("Starting new manual image backup schedule check ({curdate}).".format(curdate=curdate ), file=log_file )
print("---------------------------------------------------", file=log_file )
for tasks in runningtasks:
numtasks = numtasks + 1
if numtasks >= int(max_simbackups):
print("Number of already running tasks: {numtasks}, NOT starting more backups.".format(numtasks=numtasks ), file=log_file )
else:
print("Number of already running tasks: {numtasks}, going to start queueing backups.".format(numtasks=numtasks ), file=log_file )
print("---------------------------------------------------", file=log_file )
print("Performing check on all clients...", file=log_file )
for client in clients:
clientbackuprunning = 0
if client["lastbackup_image"]=="-" or client["lastbackup_image"] < time.time() - diff_time:
if client["online"] != False and client["lastbackup_image"]!="-" and client["lastbackup_image"]!=0:
lastbackup = datetime.datetime.fromtimestamp(client["lastbackup_image"]).strftime("%x %X")
numfailed = numfailed + 1
#If less then the maximum amount of backup tasks are already running
if numtasks <= int(max_simbackups):
#Now to find out if a backup task is already running for this client
for tasks in runningtasks:
if tasks["name"] == client["name"]:
print("Backup is already running for {clientname}, not queueing another backup for it.".format(clientname=client["name"] ), file=log_file )
clientbackuprunning = 1
if clientbackuprunning == 0:
#Now to find out if the next backup to run is a full backup or incremental backup
backups = server.get_clientimagebackups(client["id"])
lastfullbackuptime = 0
for backup in backups:
if backup["incremental"] == 0 and backup["backuptime"] > lastfullbackuptime:
lastfullbackuptime = backup["backuptime"]
#Now to check if the maximum time for a full backup has been exceeded and the backup isnt already running
if clientbackuprunning == 0:
if lastfullbackuptime < time.time() - int(max_fullimage_age):
print("Starting full image backup for {clientname}.".format(clientname=client["name"] ), file=log_file )
server.start_full_image_backup(client["name"])
else:
print("Starting incremental image backup for {clientname}.".format(clientname=client["name"] ), file=log_file )
server.start_incr_image_backup(client["name"])
print("---------------------------------------------------", file=log_file )
print("Script done executing...", file=log_file )
print("Number of image backups older then {calc_hours} hours: {numfailed}".format(numfailed=numfailed, calc_hours = diff_time / 3600 ), file=log_file )
print("---------------------------------------------------", file=log_file )
1 Like