I have looked and the log is correct the directory does not exist anywhere.
I have cleared the basebases by deleting everything and running a fresh install, the server runs for a 1 cleanup cycle without a problem and then crash happens again.
Any thoughts on how to both recover and prevent this?
here is the gdb trace
I am not familiar with the utility so I have provided everything and if I should have run it differently please advise where I went wrong.
> gdb --args urbackupsrv remove-unknown
> GNU gdb (Raspbian 7.7.1+dfsg-5) 7.7.1
> Copyright (C) 2014 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "arm-linux-gnueabihf".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from urbackupsrv...(no debugging symbols found)...done.
> (gdb) run
> Starting program: /usr/sbin/urbackupsrv remove-unknown
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
> Cannot access memory at address 0x0
> Program received signal SIGILL, Illegal instruction.
> 0x76ab1de8 in ?? () from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0
> (gdb) bt
> #0 0x76ab1de8 in ?? () from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0
> #1 0x76aae4b4 in OPENSSL_cpuid_setup () from /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.0.0
> #2 0x76fdeffc in call_init (l=<optimized out>, argc=2, argv=0x7efffc84, env=0x7efffc90) at dl-init.c:78
> #3 0x76fdf0d8 in _dl_init (main_map=0x76fff958, argc=2, argv=0x7efffc84, env=0x7efffc90) at dl-init.c:126
> #4 0x76fcfd84 in _dl_start_user () from /lib/ld-linux-armhf.so.3
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
> (gdb)
I do not really know gdb so here is what I got. If I need to do anything different please let me know.
I have taken the links out so that I can post.
thanks
gdb --args urbackupsrv remove-unknown
GNU gdb (Raspbian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
Find the GDB manual and other documentation resources online at:
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from urbackupsrv...(no debugging symbols found)...done.
(gdb) handle SIGILL nostop noprint
Signal Stop Print Pass to program Description
SIGILL No No Yes Illegal instruction
(gdb) run
Starting program: /usr/sbin/urbackupsrv remove-unknown
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
[New Thread 0x7649b440 (LWP 1984)]
2016-09-11 18:55:18: Going to remove all unknown files and directories in the urbackup storage directory. Waiting 20 seconds...
2016-09-11 18:55:38: Shutting down all database instances...
2016-09-11 18:55:38: Opening urbackup server database...
2016-09-11 18:55:38: SQLite: recovered 1051 frames from WAL file /var/urbackup/backup_server.db-wal code: 283
2016-09-11 18:55:38: SQLite: recovered 422 frames from WAL file /var/urbackup/backup_server_files.db-wal code: 283
2016-09-11 18:55:38: SQLite: recovered 1 frames from WAL file /var/urbackup/backup_server_link_journal.db-wal code: 283
2016-09-11 18:55:38: SQLite: recovered 8 frames from WAL file /var/urbackup/backup_server_settings.db-wal code: 283
[New Thread 0x75cbe440 (LWP 1985)]
[New Thread 0x759ff440 (LWP 1986)]
[New Thread 0x758ff440 (LWP 1987)]
2016-09-11 18:55:38: Testing if backup destination can handle subvolumes and snapshots...
Create subvolume '/media/backup/urbackup/testA54hj5luZtlorr494/A'
Create a snapshot of '/media/backup/urbackup/testA54hj5luZtlorr494/A' in '/media/backup/urbackup/testA54hj5luZtlorr494/B'
Transaction commit: at the end
Delete subvolume '/media/backup/urbackup/testA54hj5luZtlorr494/A'
Transaction commit: at the end
Delete subvolume '/media/backup/urbackup/testA54hj5luZtlorr494/B'
TEST OK
2016-09-11 18:55:41: Backup destination does handle subvolumes and snapshots. Snapshots enabled.
2016-09-11 18:55:41: Transitioning urbackup server database to different journaling mode...
2016-09-11 18:55:42: Cleaning up 0 percent
2016-09-11 18:55:42: Cleaning up 0 bytes on backup storage
2016-09-11 18:55:42: Database cache size is 200 MB
2016-09-11 18:55:42: Starting cleanup...
2016-09-11 18:55:42: Freeing database connections...
[New Thread 0x755ff440 (LWP 2000)]
2016-09-11 18:55:42: Enough free space now.
2016-09-11 18:55:42: Deleting incomplete file backup ( id=1, backuptime=2016-09-08 21:15:26, path=160908-2214 ) from client "CornerNet-Desktop" ( id=1 ) ...
Transaction commit: at the end
ERROR: error accessing '/media/backup/urbackup/CornerNet-Desktop/160908-2214'
2016-09-11 18:55:42: ERROR: No permission to access "/media/backup/urbackup/CornerNet-Desktop/160908-2214"
2016-09-11 18:55:42: WARNING: Warning: Directory doesn't exist: "/media/backup/urbackup/CornerNet-Desktop/160908-2214"
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x755ff440 (LWP 2000)]
0x00271c70 in ?? ()
(gdb) bt
#0 0x00271c70 in ?? ()
#1 0x000ee2f4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
You say you have cleared the databases and installed the server again.
If that’s the case then where does it get the path from in order to attempt to delete it?
You must be missing something…
In fact, how about you create an empty folder and let the cleanup remove it…?
This is a very large stretch, and I don’t think would hurt you if you did it (it might hurt uroni because he wouldn’t get logs that he probably wants) but couldn’t you just run this:sudo rm -f /media/backup/backup/CornerNet-Desktop/160908-2214
Then run your command urbackupsrv remove-unknown afterwards?
What happens if you try to create the folder it says isn’t there?
Does it say you can’t because it already exists?
If so then it could be hidden…? Can you see hidden folders?
I have created the folder and rerun the clean up. The cleanup now spots the folder but becuase I am running the backup in a BTRFS partition it tries to clean it but as it is not a sub volume it can’t (yep, I will look at how to create a sub volume next).
I have pooked in the database and the only one referancing this location is the file
backup_server_files.db
I have tried applying this
But it has not helped.
If the subvolume does not fix it I may have to try removing the entries from that Dbase file.
Right I created the subvolume OK.
I am ran remove unknown.
remove-unknown found and deleted the subvolume and then crashed.
I have checked the dbases and the directory is still listed so urbackup is not updating the deletion from the dbase.
So right now the only option is to remove the entry by hand!
You can try this, but be warned that it will remove EVERYTHING from your urbackup directory.
# Stop UrBackup service first (I don't know the actual command to do so)
sudo cp /path/to/urbackupserver/urbackup/*.db /path/to/urbackupserver/urbackup/tmp # Backup your .db files
sudo rm -f /path/to/urbackup/urbackupserver/urbackup/*.db # Make sure to specify. Don't want you deleting anything you absoultely need.
# You will need to go and set your backup location now
sudo urbackupsrv remove-unknown # Attempt to run the clean after your databases have been altered.
# Service should now start back up after running the remove-unknown