ZFS Replication not utilizing 10G Network

[wajahatahmadkhan]

So i have 2 FREENAS machines, One is the Primary storage and the second one is going to be its backup. I want to backup all the data (52TB) to the Backup machine and then keep it in sync with the primary server. After researching ZFS replication seemed to be the most suitable solution.

Ive set it up and it is working but it does not seem to fully utilize the 10G throughput. Both machines have 10G cards. I've tried separately accessing the pools from my Workstation and im able to get around 700 mbps Read/write speeds from both the machines but ZFS replications seems to be capped at 125 mbps. Im using netdata to monitor the speeds.

Sending Server speeds

Receiving Server Speeds

Is it some kind of ZFS replication limitation or another bottle neck. Here is the settings im using and ive set no limits myself

Please advise what to do cause at this speeds the initial backup will take almost a month and then keeping them in sync will also be a cumbersome task.

 

[MikeyG]

Have you tried turning off encryption yet? Assuming it's on a LAN you have control over or aren't too worried about security wise.

 

[wajahatahmadkhan]

Not

Have you tried turning off encryption yet? Assuming it's on a LAN you have control over or aren't too worried about security wise.

not yet, will give it a shot