MacOS client 2022

client
#1

Thread to track macOS client status.

Current versions:

2.4.x

https://beta.urbackup.org/macos/UrBackup%20Client%202.4.12-2.pkg

2.5.x (beta branch)

https://beta.urbackup.org/macos/UrBackup%20Client%202.5.16.pkg

Current todo list for the macOS client:

  • :green_square: Fat-binaries/separate installer for Apples ARM64 transition

Nice to have:

Please test! Thanks.

1 Like
MacOS client 2021
#2

Latest changes (with 2.4.12-2/2.5.16):

  • @Moisie reworked macOS excludes. They are now done in code
  • @ravenclaw900 Fixed scaling issues by setting something in the app configuration
  • @Moisie adjusted excludes for Monterey
  • @Moisie removed the login-item setup as the client UI is started via launchctl
  • @Moisie added a full disk access check on launch
#3

Hi there.
Not sure if this is the best place to ask for help with the Mac clients so please let me know if not!

So, Iā€™ve got two MacBook clients backing up to a urbackup server where the filesystem is BTRFS.
Both MacBook clientsā€™ log files always say
Backing up "root" without snapshot
which I find odd given that the serverā€™s backup filesystem is BTRFS (and none of the non-mac clients use this phrasing).

I can see that subvolumes are made for the clients backups in the BTRFS filesystem by using btrfs sub list DIR, so I donā€™t know if this message is actually an issue or notā€¦

Server is urbackup 2.4.13 in a docker container.
Client version is 2.5.16 (does it matter that the client and server are different minor versions?)
Host OS is ubuntu 20.04

Thank so much.

#4

That refers to whatā€™s happening on the client machine, Iā€™m unfamiliar with Macs, have one ancient one, but itā€™s running Windows 7, so it uses VSS for snapshots. What (if any) snapshot mechanism(s) are available in MacOS?

#5

I think Big Sur onwards can take APFS snapshots? Actually I thought that time machine has some kind of snapshot capability but I could easily be wrong here.

#6

There is a nice to have, open task to integrate it with TimeMachine/APFS snapshots. See the OP

#7

Indeed - the Mac client does not currently create snapshots on the client system, hence the warning.

Yes - thatā€™s correct - but the feature is not profoundly accessible for third-party applications, without specific support from Apple.

Indeed - itā€™s on the list, but no ETA (from my end, anyway).

#8

Hi there.

With the most recent Mac client on Monterey Iā€™m finding my backups tend to fail more than 50% of the time.

17/05/22 20:10 DEBUG Reflink copying is enabled
17/05/22 20:10 DEBUG Reflink copying is enabled
17/05/22 20:10 INFO Starting unscheduled incremental file backupā€¦
17/05/22 20:10 DEBUG macbook: Connecting for filelistā€¦
17/05/22 20:10 DEBUG macbook: Waiting for filelist
17/05/22 20:10 DEBUG macbook: Connecting for filelist (async)ā€¦
17/05/22 20:22 INFO macbook: Loading file listā€¦
17/05/22 20:22 DEBUG macbook Starting incremental backupā€¦
17/05/22 20:22 INFO macbook: Calculating file tree differencesā€¦
17/05/22 20:22 INFO macbook: Creating snapshotā€¦
17/05/22 20:22 INFO macbook: Deleting files in snapshotā€¦ (5645)
17/05/22 20:23 INFO macbook: Deleting files in hash snapshotā€¦(5645)
17/05/22 20:24 INFO Waiting for metadata download stream to finish
17/05/22 20:24 DEBUG Waiting for metadata download stream to finish (attempt 0, macbook). 10s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
17/05/22 20:24 DEBUG Waiting for metadata download stream to finish (attempt 1, macbook). 20s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
17/05/22 20:24 ERROR Error starting file metadata download thread
17/05/22 20:24 ERROR Backup had an early error. Deleting partial backup.

This is a fairly typical log file. This did not happen while on Big Sur with the previous Mac client.
Brute forcing the backup retries eventually makes it work.

My Mac client is 2.5.16
My server is 2.4.13 (docker)

Any tips/advice?

#9

Hi,

Thanks for the feedback. Could you have a look at the log file on the Mac to see if thereā€™s anything of interest in there?

/Library/Logs/urbackup_client_backend.log

Thanks.

#10

Hi @Moisie

Sorry for the delay. Iā€™ve attached a sanitised log file from the start of a recent transfer that failed.

Iā€™ve also copied the server log printout for the backup to correlate timings.

I cannot see anything except that I start to see error messages in the client log at the same time as the server. Does anything jump out at you?

24/05/22 12:15 DEBUG Reflink copying is enabled
24/05/22 12:15 DEBUG Reflink copying is enabled
24/05/22 12:15 INFO Starting scheduled incremental file backupā€¦
24/05/22 12:15 DEBUG macbook: Connecting for filelistā€¦
24/05/22 12:15 DEBUG macbook: Waiting for filelist
24/05/22 12:15 DEBUG macbook: Connecting for filelist (async)ā€¦
24/05/22 12:19 INFO Backing up ā€œrootā€ without snapshot.
24/05/22 12:19 INFO Indexing of ā€œrootā€ done. 204337 filesystem lookups 0 db lookups and 0 db updates
24/05/22 12:19 DEBUG macbook: Doing backup with hashed transferā€¦
24/05/22 12:19 INFO macbook: Loading file listā€¦
24/05/22 12:19 DEBUG macbook Starting incremental backupā€¦
24/05/22 12:19 INFO macbook: Calculating file tree differencesā€¦
24/05/22 12:19 INFO macbook: Creating snapshotā€¦
24/05/22 12:20 INFO macbook: Deleting files in snapshotā€¦ (5897)
24/05/22 12:21 INFO macbook: Deleting files in hash snapshotā€¦(5897)
24/05/22 12:21 INFO Waiting for metadata download stream to finish
24/05/22 12:22 DEBUG Waiting for metadata download stream to finish (attempt 0, macbook). 10s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:22 DEBUG Waiting for metadata download stream to finish (attempt 1, macbook). 20s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:22 DEBUG Waiting for metadata download stream to finish (attempt 2, macbook). 30s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:22 DEBUG Waiting for metadata download stream to finish (attempt 3, macbook). 40s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:22 DEBUG Waiting for metadata download stream to finish (attempt 4, macbook). 50s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:22 DEBUG Waiting for metadata download stream to finish (attempt 5, macbook). 1m since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:23 DEBUG Waiting for metadata download stream to finish (attempt 6, macbook). 1m 10s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:23 DEBUG Waiting for metadata download stream to finish (attempt 7, macbook). 1m 20s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:23 DEBUG Waiting for metadata download stream to finish (attempt 8, macbook). 1m 30s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:23 DEBUG Waiting for metadata download stream to finish (attempt 9, macbook). 1m 40s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:23 DEBUG Waiting for metadata download stream to finish (attempt 10, macbook). 1m 50s since last meta-data transfer. Forcefully shutting down after 2m 20s without transfer.
24/05/22 12:23 ERROR Error starting file metadata download thread
24/05/22 12:24 ERROR Backup had an early error. Deleting partial backup.

urbackup_err.log.txt (179.1 KB)

#11

Hi @absolution

Thanks for sending the logs through.

TBH Iā€™m shooting in the dark here - but try excluding /Library/Application Support//UrBackup Client/var/urbackup/ from your backup set. This is based on these entries in your client log:

2022-05-24 12:19:59: FileSrv: Sending file (normal) urbackup/tokens_b1bg7ItHesBG9Pf4eyWE.properties metadata_id=0
2022-05-24 12:19:59: FileSrv: Mapped name: /Library/Application Support/UrBackup Client/var/urbackup/data/tokens_b1bg7ItHesBG9Pf4eyWE.properties
...
2022-05-24 12:20:59: FileSrv: 1 min Timeout deleting Buffers (1024 KB) and waiting 1h more...

These correspond with the start of the server log entries, saying itā€™s waiting for the metadata download stream to finishā€¦

#12

Hi @Moisie

Thanks for the info. Iā€™ll give this a try and report back.

#13

Hi again @Moisie
Unfortunately, adding /Library/Application Support/UrBackup Client/var/urbackup/ to my excludes did not work.
Please let me know if you think of something else I could try.

#14

Yep, I have also tried adding mentioned code in the excludes. However, it did not work. Another approach is needed.

#15

Hi @absolution and @sandersbud4

Thanks for the feedback; can you please confirm you are both backing up the root of the filesystem?

I donā€™t yet have any further thoughts on what to look at, but I will investigateā€¦

#16

Hi @Moisie

I am backing up the root filesystem. I have added some exclusions, which are:
/Users/USERNAME/Library/VoiceTrigger;/Applications/Xcode.app;/Library/Developer/CommandLineTools;/Applications/Divinity - Original Sin 2.app;/Applications;/Users/USERNAME/Library/Mail;/private/var/db/SystemKey;/Library/Application Support/UrBackup Client/var/urbackup/

I mostly avoid a few items that are very large and tended to slow the backup too much (the slowness came from me using an SMR disk)

1 Like
#17

Please let us know as soon as possible after reaching a conclusion. I will be waiting.

#18

Hi @absolution

Sorry - Iā€™ve only just noticed that you wrote this:

If youā€™re using server version 2.4, you should also use client version 2.4. I suggest you uninstall the 2.5 client from your Mac, and install 2.4 client instead - link at the top of this thread.

@sandersbud4 - I donā€™t suppose this would apply to you too? :pray:

#19

Hi @Moisie
I did try this awhile ago, but the server then assumed my mac was a new client, and did not associate the backup history to it. Is it possible to roll back the version while keeping my backups linked to my Mac?

I should also add that Iā€™ve been using this client version since before I updated my Mac to its latest OS version, and did not have this issue until now.

#20

Hi @absolution

I might be wrong, but I think:

  • If you just uninstall and reinstall but keep the Application Support folder in place, all your settings should be preserved - so I would expect it to identify itself as the same machine;
  • If not, then as long as the Computer Name is the same and youā€™re using the same Internet Authentication key, I believed the server will see the client as the same as before.

Of course, YMMVā€¦

Indeed - but from the logs, I canā€™t see anything amiss at the ā€˜purely clientā€™ end of things. I think itā€™s more likely the communication between client and server thatā€™s falling over here.