Linux Debian / Ubuntu - add-backupdir CLI issue

Kabush · July 5, 2022, 10:53am

Hello,

First of all, I’d like to give thumbs up for this project and congratulate Uroni and the team for all their work.
As you might have guessed, I’m also posting in the forums because I face a problem that do not seem to be on my side but more linux client related.
We mainly operate Debian and Ubuntu server distros and most of clients or running Linux and Windows OS, we have a couple of MacOS as well but I didn’t try the client yet.

Here is the thing. Linux client can’t add new folders on their side. I have double checked that they are allowed to do it and it “works” on Debian but not at all on Ubuntu.

I took the time to try different Ubuntu versions from 20.04 up to latest LTS 22.04. Here are the client logs:

SERVER SPEC

Distributor ID: Debian
Description:    Debian GNU/Linux 11 (bullseye)
Release:        11
Codename:       bullseye
Kernel:         Linux 5.10.0-15-amd64 x86_64

UrBackup Server v2.5.23.0

CLIENT SPEC

Installed from preconfigured package given by server

Distributor ID: Ubuntu
Description:    Ubuntu 21.10
Release:        21.10
Codename:       impish
Kernel:         Linux 5.13.0-52-generic x86_64

UrBackup Client Controller v2.5.16.0

urbackupclientctl add-backupdir -o --path /home/user1/Pictures

CLIENT LOG

2022-07-04 16:31:52: No remaining buf
2022-07-04 16:31:52: rc=0 hasError=true state=0
2022-07-04 16:31:52: SERVICE_COMMANDS finished
2022-07-04 16:32:11: urbackupserver: No available slots... starting new Worker
2022-07-04 16:32:11: ClientService cmd: GET BACKUP DIRS#pw=obHlxgqQiwsNh66uvVv93RdyoSpTRj
2022-07-04 16:32:11: rc=0 hasError=true state=0
2022-07-04 16:32:11: ClientService cmd: SAVE BACKUP DIRS#pw=obHlxgqQiwsNh66uvVv93RdyoSpTRj&all_virtual_clients=1&enable_client_paths_use=1
2022-07-04 16:32:11: rc=0 hasError=true state=0

urbackupclientctl list-backupdir
No directories are being backed up

Sending the command multiple time does not change behavior


Distributor ID: Ubuntu
Description:    Ubuntu 22.04 LTS
Release:        22.04
Codename:       jammy
Kernel:         Linux 5.15.0-40-generic x86_64

urbackupclientctl add-backupdir -o --path /home/user1/Pictures

CLIENT LOG

2022-07-04 17:07:16: No remaining buf
2022-07-04 17:07:26: ClientService cmd: GET BACKUP DIRS#pw=obHlxgqQiwsNh66uvVv93RdyoSpTRj
2022-07-04 17:07:26: rc=0 hasError=true state=0
2022-07-04 17:07:26: ClientService cmd: SAVE BACKUP DIRS#pw=obHlxgqQiwsNh66uvVv93RdyoSpTRj&all_virtual_clients=1&enable_client_paths_use=1
2022-07-04 17:07:26: rc=0 hasError=true state=0


urbackupclientctl list-backupdir
No directories are being backed up

Sending the command multiple time does not change behavior

Distributor ID: Debian
Description:    Debian GNU/Linux bookworm/sid
Release:        testing
Codename:       bookworm
Kernel:         Linux 5.18.0-2-amd64 x86_64

rBackup Client Controller v2.5.16.0


urbackupclientctl add-backupdir -o --path /home/user/Music

CLIENT LOG

2022-07-04 16:36:08: ClientService cmd: GET BACKUP DIRS#pw=kOLhlMCqGU4jZaCebZvoXmPcdRYLxA
2022-07-04 16:36:08: rc=0 hasError=true state=0
2022-07-04 16:36:08: ClientService cmd: SAVE BACKUP DIRS#pw=kOLhlMCqGU4jZaCebZvoXmPcdRYLxA&all_virtual_clients=1&enable_client_paths_use=1&dir_0=%2Fhome%2Fuser1%2FPictures&dir_0_name=Pictures/follow_symlinks,symlinks_optional,share_hashes&dir_0_group=0&dir_1=%2Fhome%2Fuser1%2FVideos&dir_1_name=Videos/follow_symlinks,symlinks_optional,share_hashes&dir_1_group=0&dir_2=%2Fhome%2Fuser1%2Fgitlab&dir_2_name=gitlab/optional,follow_symlinks,symlinks_optional,share_hashes&dir_2_group=0&dir_3=%2Fhome%2Fuser1%2FMusic&dir_3_name=Music/optional,follow_symlinks,symlinks_optional,share_hashes&dir_3_group=0
2022-07-04 16:36:08: rc=0 hasError=true state=0
2022-07-04 16:36:08: Started connection to SERVICE_FILESRV
2022-07-04 16:36:08: FileSRV received 73
2022-07-04 16:36:08: FileSrv: Received data...
2022-07-04 16:36:08: FileSrv: Received a Packet.
2022-07-04 16:36:08: FileSrv: Sending file (normal) urbackup/settings.cfg metadata_id=0
2022-07-04 16:36:08: FileSrv: Mapped name: /usr/local/var/urbackup/data/settings.cfg
2022-07-04 16:36:08: FileSRV received 21
2022-07-04 16:36:08: FileSrv: Received data...
2022-07-04 16:36:08: FileSrv: Received a Packet.
2022-07-04 16:36:08: Received flush.
2022-07-04 16:36:08: FileSrv: Recv Error in RecvMessage
2022-07-04 16:36:08: SERVICE_FILESRV finished
2022-07-04 16:36:08: Proxy buf size 76 content: HTTP/1.0 200 Connection Established
Proxy-agent: Apache/2.4.53 (Debian)

After executing the command 5 to 10 times, it gets added like expected.

urbackupclientctl list-backupdir      

PATH                   NAME     FLAGS                                                   
---------------------- -------- ------------------------------------------------------- 
/home/user1/Pictures Pictures follow_symlinks,symlinks_optional,share_hashes          
/home/user1/Videos   Videos   follow_symlinks,symlinks_optional,share_hashes          
/home/user1/gitlab   gitlab   optional,follow_symlinks,symlinks_optional,share_hashes 
/home/user1/Music    Music    optional,follow_symlinks,symlinks_optional,share_hashes

All clients are configured “internet client” and forward proxy is working fine.
I followed Connect clients with a HTTPS CONNECT web proxy | UrBackup Developer Blog for the proxy part. The web interface is proxyfied with a reverse proxy on fastCGI and it works great.

For linux clients, if I add path via web, they get updated and backups are working as well.
I also installed the linux GUI on top of the pre-configured client and adding directory works.
I’m wondering what’s wrong with the linux CLI (witch is auto-updatable and that is what we aim for).

Windows Clients work like a charm in both way, adding folder through client or web work as expected.

Do anyone has a clue of what could go wrong? Kernel Version? Missing library?
Firewall is open on 55415 as well a 443 (we don’t need UDP broadcasts for discovery) and once again the issue is only about urbackupclientctl add-backupdir.

Since this post is quite extended already, may I ask if any update on the LDAP part is planned any time soon? We have multiple service already relying on LDAP but I must admit that on urbackup, it’s not a peace of cake

If you need more logs and / or have other solutions the debug even deeper, I will do what is necessary to comply and send the information needed.

Thank you for your time, and again, thank you all for this great project !

Best regards,

Kabush · July 7, 2022, 2:38pm

UPDATE: Server version is now 2.5.24, client version is still 2.5.16 (server didn’t auto-update the clients, doing it manually)

Now using websocket for internet client following this post Server 2.5.24/Client 2.5.18

Command urbackupclientctl add-backupdir --path /some/path/ still not working on client version 2.5.16 - 2.5.17 and 2.5.18 tested on Ubuntu 22.04 LTS

2022-07-07 16:32:31: urbackupserver: No available slots... starting new Worker
2022-07-07 16:32:31: ClientService cmd: GET BACKUP DIRS#pw=OCuyQN7QHnj7u8jStYPTshuUfqNA
2022-07-07 16:32:31: rc=0 hasError=true state=0

uroni · July 10, 2022, 10:14pm

Thanks for the detailed report and retesting with the new version.

2.5.19 should fix the problem. Could you retest with that?

Kabush · July 11, 2022, 12:16pm

Hello @uroni !

Thank you os much for this quick fix ! Tested client v2.5.19 on:

Ubuntu 20.04.4 LTS
kernel 5.4.0-121-generic x86_64
---
Ubuntu 22.04 LTS
kernel 5.15.0-40-generic x86_64
---
Debian 12 bookworm/sid
kernel 5.18.0-2-amd64 x86_64

And it works like a charm ! You did great job here !

uroni · July 11, 2022, 8:19pm

The main problem is that I have not much knowledge about the kinds of LDAP setups around. Also, I don’t think anymore that it is a good idea to allow LDAP admin access (this is the kind of setup that would allow an attacker to delete your backups and your primaries). For self-service file access in backups it’s still usefull…

Kabush · July 12, 2022, 8:03am

Yes, I understand and must agree with the fact that LDAP admin access is not the most secured way of doing things.

Indeed, that was my only wondering, how could I allow user access for self-service… and that’s where LDAP would have been handy.

On linux there is still the CLI urbackupclientctl browse and urbackupclientctl restore-start who does it pretty well. One of my user complained that he could not see a list of files contained in a backup while using the command line. Would that be possible to do or the re-work is just too big to consider this new feature ?

Should I open a new topic to ask that feature and see if the community would appreciate it?

Kind regards,