Hi,
I'm currently running into performance issues during replication of datasets between two hyperconverged truenas boxes. I've already checked the usual topics. Here's a quick summary:
Source machine:
Now to the problem: When I start a replication task using SSH, transfer speeds of the replication are unfortunately limited to ~61 MiB/s. CPU at the source machine is below 30% according to htop with several active users. On the backup side, ssh is using 95% of a core while the second core is idling. What confuses me is that the CPU supports Intel Quick Assist or AES-NI respectively and is also detected within TrueNAS as "dmesg | grep aes" reports "aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA-1,SHA-256> on motherboard". Cipher settings were left on default during setup of the ssh connection.
Which cipher suite is used by TrueNAS? Are there any possibilities to accelerate the transfer?
I'm currently running into performance issues during replication of datasets between two hyperconverged truenas boxes. I've already checked the usual topics. Here's a quick summary:
Source machine:
- Latest Truenas running within latest ESXi 7.0
- E5-1650 v4, 2 vCPUs for Truenas
- Supermicro X10SRH-CLN4F, SAS3-controller passed through to TrueNAS via vt-d
- SAS3-EL1 Backplane
- 64 GB DDR4 ECC RAM, 32GB reserved for TrueNAS
- Boot-Datastore running from Intel M.2 SSD
- 6x 8TB Seagate IronWolf in Raid-z2, 1 vdev
- Intel x520-da2 NIC connected via DAC and 10gbe Switch
- Latest Truenas running within latest ESXi 7.0
- C3758, 2 vCPUs for Truenas
- Supermicro A2SDi-8C+-HLN4F, SATA-controllers passed through to TrueNAS via vt-d
- SATA3-Passthrough Backplane
- 8GB DDR4 ECC RAM, 4GB reserved for TrueNAS
- Boot-Datastore running from Intel M.2 SSD
- 5x 8TB Seagate IronWolf in Raid-z1, 1 vdev
- Intel x520-da1 NIC connected via DAC and 10gbe Switch
Now to the problem: When I start a replication task using SSH, transfer speeds of the replication are unfortunately limited to ~61 MiB/s. CPU at the source machine is below 30% according to htop with several active users. On the backup side, ssh is using 95% of a core while the second core is idling. What confuses me is that the CPU supports Intel Quick Assist or AES-NI respectively and is also detected within TrueNAS as "dmesg | grep aes" reports "aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA-1,SHA-256> on motherboard". Cipher settings were left on default during setup of the ssh connection.
Which cipher suite is used by TrueNAS? Are there any possibilities to accelerate the transfer?