I'm at a complete loss here. Recently installed a Chelsio T420-CR to boost NFS performance. Been noticing that the transmit speed degrades over time; dropping from 6-9Gbps to 60-400Mbps
Current setup:
FreeNAS 11.1U6 / 32GB RAM /Chelsio T420-CR
Jumbo Frames enabled / Direct connect to workstation (i.e. no switches)
TSO, LRO Enabled
WORKSTATION
Ubuntu 18.04 / 32GB RAM / Intel X520-SR
TSO, LRO Disabled due to known driver bug
Performance:
After a clean reboot of both FreeNAS and workstation, iperf results:
FN -> Workstation ~6Gbps
Workstation -> FN ~9.88Gbps
24 hours later, noticed dramatic pauses and slowdown with NFS
FN -> Workstation ~200-400Mbps
Workstation -> FN ~9.88Gbps
FreeNAS still receives packets at near line speed, but transmission speed dropped significantly. At first I thought maybe the problem was on workstation end, so I rebooted the workstation, but the transfer numbers didn't change.
Rebooting FreeNAS brought performance back to 6Gbps / 9.88Gbps. I've made various changes to rule out hardware issues. Disabling jumbo frames, disabling TSO and LRO on the Chelsio, changed TCP window sizes, changing tcp congestion algo from default newreno to htcp, etc... Nothing made a difference. I've tuned the network parameters based on the various 10GB tuning guides out there (including the ones posted in this forum)
Anyone with experience using these Chelsio's? Any idea why FN performance would change so much in a matter of 24 hours?
Current setup:
FreeNAS 11.1U6 / 32GB RAM /Chelsio T420-CR
Jumbo Frames enabled / Direct connect to workstation (i.e. no switches)
TSO, LRO Enabled
WORKSTATION
Ubuntu 18.04 / 32GB RAM / Intel X520-SR
TSO, LRO Disabled due to known driver bug
Performance:
After a clean reboot of both FreeNAS and workstation, iperf results:
FN -> Workstation ~6Gbps
Workstation -> FN ~9.88Gbps
24 hours later, noticed dramatic pauses and slowdown with NFS
FN -> Workstation ~200-400Mbps
Workstation -> FN ~9.88Gbps
FreeNAS still receives packets at near line speed, but transmission speed dropped significantly. At first I thought maybe the problem was on workstation end, so I rebooted the workstation, but the transfer numbers didn't change.
Rebooting FreeNAS brought performance back to 6Gbps / 9.88Gbps. I've made various changes to rule out hardware issues. Disabling jumbo frames, disabling TSO and LRO on the Chelsio, changed TCP window sizes, changing tcp congestion algo from default newreno to htcp, etc... Nothing made a difference. I've tuned the network parameters based on the various 10GB tuning guides out there (including the ones posted in this forum)
Anyone with experience using these Chelsio's? Any idea why FN performance would change so much in a matter of 24 hours?