Can't make backups

Server
1

Hi All,

I have UrBackup Server 2.3.7 on NAS Asus as1002T (from app store). Windows clients backuping with errors (Win 2012 server R2, urbackup client 2.3.4):

backupcom->AddToSnapshotSet(&(Server->ConvertToWchar(selected_vols[i])[0]), GUID_NULL, &additional_refs[i].volid) failed. VSS error code VSS_E_UNEXPECTED_PROVIDER_ERROR
VSS provider information:
��� ���⠢騪�: “Microsoft File Share Shadow Copy provider”
��� ���⠢騪�: ���� �����
Id ���⠢騪�: {89300202-3cec-4981-9171-19f59559e0f2}
�����: 1.0.0.1
��� ���⠢騪�: “Microsoft Software Shadow Copy provider 1.0”
��� ���⠢騪�: ���⥬��
Id ���⠢騪�: {b5946137-7b9f-4925-af80-51abd60b20d5}
�����: 1.0.0.7
Creating snapshot of “Arkhive” failed.
and etc [skip]

Client Centos7 (urbackup client 2.3.4 without snapshots, md raid1):

Client calculated hash of “/volume1/UrBackup/webhosting2/190913-1904/urbackup_backup_scripts/mariadbxtrabackup/team_db/ef_strih_codes.ibd” differs from server calculated hash. This may be caused by a bug or by random bit flips on the client or server hard disk. Failing backup. (Hash: sha512-nosparse, client hash: RdOAzvZCE237PyWKDvrZ2l5aO6NRhHWocAsIf1yVe5hLOYB1xlIDSu9pM+djrE6DoJzP9idiPDTNydLM+5FfQA==, server hash: nkXZA187xGdUFQboqhVMEcMS2ya9BoX+kCMDTtsHqxV4Qat+zxvFJSFkxs0+oPmm8KT2ygtEfFNFvmLIj5ISoA==)
Client calculated hash of “/volume1/UrBackup/webhosting2/190913-1904/urbackup_backup_scripts/mariadbxtrabackup/lustory_old/b_sale_fuser.ibd” differs from server calculated hash. This may be caused by a bug or by random bit flips on the client or server hard disk. Failing backup. (Hash: sha512-nosparse, client hash: AobWySupYRTiIdw9gfjjtpAEZjZh70bzM3AUvnGRETI8Bb3hlgbtWEiLtaV+Zy9ntR3fbaisGXJFn09LHohHLQ==, server hash: gOq3lJ4y29kpfkWFrpvhQNbmoQVSEIN1xMDXMd942GaB/UJ/g5PL+lTiCZ6LyAvYLqW+hKfGEALewIGK3Y55Mg==)
FATAL: Backup failed because of disk problems (see previous messages)
Backup failed

Any suggestions?

Thanks

2

As far as I understand, the cause of errors on Windows machines is encrypted disks via VeraCrypt. Is it safe for backups?

3

See here: UrBackup - No shadow copies on TrueCrypt/VeraCrypt volumes
If backing up without snapshot is okay, depends on the data. It won’t be able to backup some open files, for example.

That definitely shouldn’t happen. If you can, could you retry that with the most recent versions of urbackup 2.4.x at Testing - UrBackup - Discourse …
Also please tell me the exact version of MariaDB and the backup tool you used (mariabackup?)

4

Thank you for reply. I make backup of static content (compressed database dump files and scripts). I think there should be no problems with them

Glad tohelp with testing. I’ll try to test 2.4.x version tonight.
MariaDB version:
mysql Ver 15.1 Distrib 10.1.31-MariaDB, for Linux (x86_64) using readline 5.1
Percona-xtrabackup on this server:
percona-xtrabackup.x86_64 2.3.6-1.el7

BTW, Urbackup server provide client version of 2.1.17 (instead of 2.3.x, this version I’ve installed manually). I’m planning to test this version too…

Thanks

5

I setup a centos with the same versions you have and I couldn’t reproduce the problem yet. Could you tell me how large e.g. ef_strih_codes.ibd is?

6

ls -la ./ef_strih_codes.ibd
-rw-rw---- 1 mysql mysql 1673527296 Sep 15 20:01 ./ef_strih_codes.ibd

My apologies, I didn’t test yet 2.4.x urbackup client. I’m waiting finish of “Start-up recovery” process:
15.09.19 09:42 |WARNING |Deleting file backup [id=3, path=190912-1107, backuptime=2019-09-12 08:07:54, clientid=5, client=webhosting2]…
15.09.19 12:48 |WARNING |Deleting file backup [id=8, path=190912-2354, backuptime=2019-09-12 20:54:43, clientid=5, client=webhosting2]…
15.09.19 17:16 |WARNING |Deleting file backup [id=9, path=190913-1904, backuptime=2019-09-13 16:04:55, clientid=5, client=webhosting2]…
15.09.19 20:24 |WARNING |Deleting file backup [id=12, path=190914-1904, backuptime=2019-09-14 16:04:49, clientid=5, client=webhosting2]…

After finish, I’ll try to test 2.4.x

BTW, I forgot to tell that I’m using “Internet Backup”, because this is remore host

7

Thanks, a client debug log file would perhaps have some hints as well (see Having problems with UrBackup? Please read before posting on how to create one)

8

Sure, will do and send results here.

Thanks

9

Now, I’m trying to make backup with urbackup client 2.1.17 (CentOS release 6.8 (Final),
mysql Ver 15.1 Distrib 10.1.16-MariaDB, for Linux (x86_64) using readline 5.1, percona-xtrabackup-22.x86_64)

Errors
17.09.19 03:04
mariadbxtrabackup: /usr/local/etc/urbackup/mariadbxtrabackup.conf: line 14: mysqld: command not found
Errors
17.09.19 03:04
mariadbxtrabackup: Starting binary backup of MariaDB at Tue Sep 17 03:04:58 MSK 2019…
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup version 2.2.13 based on MySQL server 5.6.24 Linux (x86_64) (revision id: 70f4be3)
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: uses posix_fadvise().
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: cd to
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: open files limit requested 0, set to 1024
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: using the following InnoDB configuration:
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: innodb_data_home_dir = ./
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: innodb_log_group_home_dir = ./
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: innodb_log_files_in_group = 2
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup: innodb_log_file_size = 47185920
Errors
17.09.19 03:04
mariadbxtrabackup: 2019-09-17 03:04:58 7f9c77474720 InnoDB: Operating system error number 2 in a file operation.
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: The error means the system cannot find the path specified.
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: File ./ib_logfile0: ‘open’ returned OS error 71. Cannot continue operation
Errors
17.09.19 03:04
mariadbxtrabackup: 2019-09-17 03:04:58 7f9c77474720 InnoDB: Assertion failure in thread 140309992785696 in file os0file.cc line 658
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: We intentionally generate a memory trap.
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: If you get repeated assertion failures or crashes, even
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: immediately after the mysqld startup, there may be
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: corruption in the InnoDB tablespace. Please refer to
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
Errors
17.09.19 03:04
mariadbxtrabackup: InnoDB: about forcing recovery.
Errors
17.09.19 03:04
mariadbxtrabackup: 00:04:58 UTC - xtrabackup got signal 6 ;
Errors
17.09.19 03:04
mariadbxtrabackup: This could be because you hit a bug or data is corrupted.
Errors
17.09.19 03:04
mariadbxtrabackup: This error can also be caused by malfunctioning hardware.
Errors
17.09.19 03:04
mariadbxtrabackup: We will try our best to scrape up some info that will hopefully help
Errors
17.09.19 03:04
mariadbxtrabackup: diagnose the problem, but since we have already crashed,
Errors
17.09.19 03:04
mariadbxtrabackup: something is definitely wrong and this may fail.
Errors
17.09.19 03:04
mariadbxtrabackup:
Errors
17.09.19 03:04
mariadbxtrabackup: Thread pointer: 0x0
Errors
17.09.19 03:04
mariadbxtrabackup: Attempting backtrace. You can use the following information to find out
Errors
17.09.19 03:04
mariadbxtrabackup: where mysqld died. If you see no messages after this, something went
Errors
17.09.19 03:04
mariadbxtrabackup: terribly wrong…
Errors
17.09.19 03:04
mariadbxtrabackup: stack_bottom = 0 thread_stack 0x10000
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup(my_print_stacktrace+0x2e) [0x8b7ace]
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup(handle_fatal_signal+0x273) [0x729863]
Errors
17.09.19 03:04
mariadbxtrabackup: /lib64/libpthread.so.0(+0xf7e0) [0x7f9c770577e0]
Errors
17.09.19 03:04
mariadbxtrabackup: /lib64/libc.so.6(gsignal+0x35) [0x7f9c75673495]
Errors
17.09.19 03:04
mariadbxtrabackup: /lib64/libc.so.6(abort+0x175) [0x7f9c75674c75]
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup() [0x601745]
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup(os_file_create_func(char const*, unsigned long, unsigned long, unsigned long, unsigned long*)+0x218) [0x603778]
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup() [0x589a09]
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup(main+0x91c) [0x58df1c]
Errors
17.09.19 03:04
mariadbxtrabackup: /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f9c7565fd1d]
Errors
17.09.19 03:04
mariadbxtrabackup: xtrabackup() [0x582839]
Errors
17.09.19 03:04
mariadbxtrabackup:
Errors
17.09.19 03:04
mariadbxtrabackup: Please report a bug at https://bugs.launchpad.net/percona-xtrabackup
Errors
17.09.19 03:04
Script “mariadbxtrabackup” returned a nun-null value 1. Failing backup.
Errors
17.09.19 03:04
Client webhosting1 went offline.
Errors
17.09.19 03:05
Error saving metadata. Could not open output file at “/volume1/UrBackup/webhosting1/190917-0225/.hashes/urbackup_backup_scripts/mariadbxtrabackup/.dir_metadata”
Errors
17.09.19 03:14
Backup failed

I’ve edited 14-th line with absoulutely path. May be it will helpful to use path to mysqld via variable like:
MYSQLD=`which mysqld` or `whereis mysqld`

Now I’m waiting end of full file backup…

10

I’ve tried to install beta client version on Centos 7:

TF=mktemp && wget “http://beta.urbackup.org/Client/2.4.6%20beta/UrBackup%20Client%20Linux%202.4.6%20beta.sh” -O $TF && sudo sh $TF; rm $TF
–2019-09-17 11:56:06-- http://beta.urbackup.org/Client/2.4.6%20beta/UrBackup%20Client%20Linux%202.4.6%20beta.sh
Resolving beta.urbackup.org (beta.urbackup.org)… 2a01:488:42:1000:50ed:84c3:fff0:6c31, 80.237.132.195
Connecting to beta.urbackup.org (beta.urbackup.org)|2a01:488:42:1000:50ed:84c3:fff0:6c31|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 21015017 (20M) [application/x-sh]
Saving to: ‘/tmp/tmp.GLNStB3RCQ’

100%[======================================>] 21,015,017 41.1MB/s in 0.5s

2019-09-17 11:56:07 (41.1 MB/s) - ‘/tmp/tmp.GLNStB3RCQ’ saved [21015017/21015017]

Verifying archive integrity… All good.
Uncompressing UrBackup Client Installer for Linux 100%
Installation of UrBackup Client 2.4.6 beta to /usr/local … Proceed ? [Y/n]
Y
Uncompressing install data…
Assuming RedHat (derivative) system
Detected systemd
Detected architecture x86_64-linux-glibc
Glibc not installed or too old (2). Falling back to Android NDK…
Installing systemd unit…
Restarting UrBackup Client service…
Successfully started client service. Installation complete.
rm: remove regular file ‘/tmp/tmp.GLNStB3RCQ’? Y

yum list installed | grep glib
dbus-glib.x86_64 0.100-7.el7 installed
glib-networking.x86_64 2.50.0-1.el7 @base
glib2.x86_64 2.50.3-3.el7 @base
glibc.x86_64 2.17-260.el7_6.6 @updates
glibc-common.x86_64 2.17-260.el7_6.6 @updates
glibc-devel.x86_64 2.17-260.el7_6.6 @updates
glibc-headers.x86_64 2.17-260.el7_6.6 @updates
json-glib.x86_64 1.2.6-1.el7 @base

urbackupclientbackend --version
UrBackup Client Backend v2.4.6.0
Copyright (C) 2011-2019 Martin Raiber
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Looks like client was updated, but:
Glibc not installed or too old (2). Falling back to Android NDK…

11

I’ve upgraded percona-xtrabackup to 24 version and backup successfuly ended.
So, Centos 6 + urbackup client 2.1.17+MariaDB 10.1.16+percona-xtrabackup-24 looks working. I’m continue testing of 2.4.x urbackup client beta and will send results in this thread.

BTW, restore testing:

#urbackupclientctl browse
[{
“archived”: 0,
“backuptime”: 1568710263,
“id”: 24,
“incremental”: 0,
“size_bytes”: 5534698644
}
]
# urbackupclientctl browse -v incr
Error getting file backups

# ./restore-mariadbbackup
2019-09-17 13:08:35 139671459985440 [Note] Plugin ‘FEEDBACK’ is disabled.
Retrieving backups…
Error getting file backups
#

12

I suspect that issue can be placed around:

# /usr/sbin/mysqld --verbose --help | grep “datadir " | tr -s " " | cut -d” " -f2
2019-09-17 20:38:04 140353466984480 [Note] Plugin ‘FEEDBACK’ is disabled.
/var/lib/mysql/

I’ve edited lines from mariadbxtrabackup.conf config:
then
MARIADB_DATADIR=/var/lib/mysql
#MARIADB_DATADIR=`/usr/sbin/mysqld --verbose --help | grep “datadir " | tr -s " " | cut -d” " -f2`
fi

but seems it doesn’t work:

# ./restore-mariadbbackup
Retrieving backups…
Error getting file backups

# urbackupclientctl browse -v incr
Error getting file backups

13

The same on other server:

# ./restore-mariadbbackup
2019-09-18 2:24:04 139946494900480 [Warning] ‘THREAD_CONCURRENCY’ is deprecated and will be removed in a future release.
2019-09-18 2:24:04 139946494900480 [Warning] option ‘net_buffer_length’: unsigned value 16777216 adjusted to 1048576
2019-09-18 2:24:04 139946494900480 [Note] Plugin ‘FEEDBACK’ is disabled.
Retrieving backups…
Error getting file backups

Servers have only one full backup without incremental. Perhaps, it’s linked with the issue…

14

I’ve tested 2.4.6 beta urbackup client. Backup was completed without errors, but have issues with ./restore-mariadbbackup, see before.
Now, I’m testing 2.4.7 beta urbackup client.

Thanks