StorageCurious
Explorer
- Joined
- Sep 28, 2022
- Messages
- 60
Hi,
I have a replication going between two fairly powerful machines. the data transferred between them is around 600Mbits/s as recorded on the network graphs of TrueNAS.
I guess it could be worst, but I can't understand why it tops off at this speed. According to TrueNAS graphs...
- On both sides (sending and receiving) the disks are not that busy (around 20%/15%). Both sides have a pool made of multiple mirrored vdevs (on side has 6, the other 3 hence the difference in usage I would think). (7200k HDDs)
- The CPU stays under 10% usage, hovers around 2%-3%.
- The network between them is a 10Gb link, and tested at that speed with iperf3 between both machines.
- The receiving side does not have a SLOG, but past experience with another set of machines tells me if doesn't make a difference for replication as the SLOG didn't seem used for the replication.
I have tried replicating to another machine in the past, with the same sort of performance.
Is this simply sort sort of "cost-of-ZFS" thing, or is there a bottleneck I forgot to consider?
I have a replication going between two fairly powerful machines. the data transferred between them is around 600Mbits/s as recorded on the network graphs of TrueNAS.
I guess it could be worst, but I can't understand why it tops off at this speed. According to TrueNAS graphs...
- On both sides (sending and receiving) the disks are not that busy (around 20%/15%). Both sides have a pool made of multiple mirrored vdevs (on side has 6, the other 3 hence the difference in usage I would think). (7200k HDDs)
- The CPU stays under 10% usage, hovers around 2%-3%.
- The network between them is a 10Gb link, and tested at that speed with iperf3 between both machines.
- The receiving side does not have a SLOG, but past experience with another set of machines tells me if doesn't make a difference for replication as the SLOG didn't seem used for the replication.
I have tried replicating to another machine in the past, with the same sort of performance.
Is this simply sort sort of "cost-of-ZFS" thing, or is there a bottleneck I forgot to consider?