Urbackup speed on amazon efs

Feature requests
1

Hello

I get very slow speed on amazon efs (70-200kb/sec) .
So far my diagnotic is that i get very small bandwith if i do very small i/o
and it goes faster as io are larger, thus as urbackup does a lot of link/unlink operation it’s slow on it

However i tested writing up to 10 files in parallel and the more files you write in parrallel , the faster you get with almost no performance hit.

This i wonder if urbackup could makes the write to the file store in parrallel ?
If there could be a write pool to manage 10 writes at a time , the backup would go 10 times faster.

The result of a script i wrote to benchmark this :

#====4 thread io=========
speed= 646K /s thread=4 bs=2k count=25 sec=0.310
speed= 1.6M /s thread=4 bs=4k count=256 sec=2.627
speed= 3.0M /s thread=4 bs=8k count=128 sec=1.379
speed= 5.1M /s thread=4 bs=16k count=64 sec=0.804
speed= 9.7M /s thread=4 bs=32k count=32 sec=0.426
speed= 14M /s thread=4 bs=64k count=16 sec=0.297
speed= 19M /s thread=4 bs=128k count=8 sec=0.222
speed= 34M /s thread=4 bs=1024k count=1 sec=0.124
speed= 54M /s thread=4 bs=2048k count=1 sec=0.153
#====10 thread io=========
speed= 1.4M /s thread=10 bs=2k count=25 sec=0.364
speed= 3.8M /s thread=10 bs=4k count=256 sec=2.699
speed= 6.3M /s thread=10 bs=8k count=128 sec=1.643
speed= 12M /s thread=10 bs=16k count=64 sec=0.900
speed= 23M /s thread=10 bs=32k count=32 sec=0.455
speed= 32M /s thread=10 bs=64k count=16 sec=0.330
speed= 44M /s thread=10 bs=128k count=8 sec=0.238
speed= 62M /s thread=10 bs=1024k count=1 sec=0.166
speed= 107M /s thread=10 bs=2048k count=1 sec=0.192
#====1 thread io=========
speed= 201K /s thread=1 bs=2k count=256 sec=2.548
speed= 431K /s thread=1 bs=4k count=2560 sec=23.788
speed= 818K /s thread=1 bs=8k count=1280 sec=12.523
speed= 1.7M /s thread=1 bs=16k count=640 sec=6.187
speed= 3.5M /s thread=1 bs=32k count=320 sec=2.953
speed= 5.9M /s thread=1 bs=64k count=160 sec=1.741
speed= 9.0M /s thread=1 bs=128k count=80 sec=1.138
speed= 15M /s thread=1 bs=1024k count=10 sec=0.685
speed= 20M /s thread=1 bs=2048k count=5 sec=0.533

2

Humm even outside amazon ,on a physical server (big 64 cores one, with huge memory)

I just started to feed the full of all all servers to the backup server.
And there a files in queue for all servers, with the transfer 100% complete
and the other ones with an empty queue , which are writting metadata

So the transfett seems to be suffisently threaded or at least it s not an issue , but the writes to the background store dont seems to be so threaded ( in this situation urbackup server seems stuck to 100-140%, which means it use about 1 core cpu)