Each of the offices has a site to site (IPSec) VPN tunnel setup between them, and are running on separate subnets. All ports are open up between the offices. Each office has a backup server that is backing up a data server on its subnet, and one on a remote subnet.
As an example:
BACKUPSVR_A1, 10.10.1.100
DATASVR_A1, 10.10.1.101
BACKUPSVR_B1, 10.10.2.100
DATASVR_B1, 10.10.2.101
The A servers can see each other without issue, and the B servers can see each other without help.
Because each data server is getting backed up to both backup servers, I needed to manually add the backup server key to the data server, as per: UrBackup - Server administration manual
See the info on server_ident.key