10Gbps Upload to S3

[whmm]

Hi all,

I am currently on a 10Gbps connection and trying to upload to S3 using a cloud sync task. I can't get over 800Mbps speeds during the upload. Internally, I am getting full 10Gbps speeds from one NAS to another, so the switch and NIC appears to be working properly. I am also getting full outbound bandwidth during various speedtests and manual uploads using the AWS CLI with the same AWS creds.

Currently the cloud sync task has default settings using PUSH and SYNC. I have also tried COPY. Under advanced options I have tried enabling --fast-list, changing transfers to 16, and setting explicit bandwidth to 10Gbps. No combinations of these settings worked. I confirmed bandwidth settings are formatted properly by reducing down to 10Mbps and seeing it throttled appropriately.

Looking for any help or advice. Resources on 10G are a bit sparse, and I couldn't find nearly anything about uploads to S3 in any docs or forum posts.

 

[MrGuvernment]

AWS have some caps that are being hit?

 

[whmm]

No caps that I'm aware of. Ultimately I uploaded using the AWS CLI and was able to utilize full bandwidth. The uploads were from the same NAS, same network, and even on the computer I was using to access the TrueNAS web interface. I did more testing and still couldn't figure out why the cloud sync task was hitting a wall. Next time I have the opportunity with a 10Gb line I'll poke around more, but the opportunities are few and far between.

 

[MrGuvernment]

Wonder if it is an app limitation? Much like how QNAP sync with OneDrive is just horrendously slooooow vs desktop client.