Loading files (metadata only) slow

Some of my backups are slow. Mostly on clients with file systems that contain a lot of small files (as expected). In most cases, UrBackup logs this before spending a lot of time being (seemingly) idle:

Loading file "[...]" (metadata only) id=[...]

… which can take a long time when the specified file is a directory which contains many files. I’m seeing this on both local and non-local (NFS) paths. For example, “Loading file” for a directory with 400 files took 14 minutes. I’m not sure if this is expected behaviour, or if UrBackup is doing heavy stuff in the background that I don’t know about.

The code (assuming I followed the correct path) at https://github.com/uroni/urbackup_backend/blob/c139b09cb59cef002c6a570848489c23fcd96418/urbackupserver/IncrFileBackup.cpp#L219 indicates that this ‘loading’ process is done by the client. What I’m seeing on the client:

  • A few syscalls: poll, and sometimes recvfrom and sendto (haven’t inspected these) - all to the UrBackup server
  • Some network traffic to the UrBackup server (using internet mode)
  • No open files except for /dev/null, the WAL DB, the client log, and network connections to the UrBackup server

Could anyone explain what Loading file "[...]" (metadata only) id=[...] entails, exactly?