I finally got fiber runs from my basement homelab to my desktop, however I found that the Tx bandwidth of my FN box was limited to ~5Gbit/s, while Rx side is OK.
192.168.20.101 is the FN box and 192.168.30.101 is my desktop.
To determine where the problem lies, I run another set of test to a FN VM (192.168.20.122)
It appears to be some headroom due to virtualization but still decent numbers. This also suggest that something is happening on the FN box, so I reboot the box into Ubuntu live CD (192.168.20.110)
The problem goes away. Was there any issue on FreeNAS with X540? As far as I know there used to be some problems with ixg drivers but that should be resolved AFAIK.
Code:
c:\iperf>iperf -c 192.168.20.101 ------------------------------------------------------------ Client connecting to 192.168.20.101, TCP port 5001 TCP window size: 208 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.30.101 port 51823 connected with 192.168.20.101 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 10.7 GBytes 9.15 Gbits/sec c:\iperf>iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 208 KByte (default) ------------------------------------------------------------ [ ID] Interval Transfer Bandwidth [ 4] local 192.168.30.101 port 5001 connected with 192.168.20.101 port 15309 [ 4] 0.0-10.0 sec 5.18 GBytes 4.43 Gbits/sec
192.168.20.101 is the FN box and 192.168.30.101 is my desktop.
To determine where the problem lies, I run another set of test to a FN VM (192.168.20.122)
Code:
c:\iperf>iperf -c 192.168.20.122 ------------------------------------------------------------ Client connecting to 192.168.20.122, TCP port 5001 TCP window size: 208 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.30.101 port 51809 connected with 192.168.20.122 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 9.24 GBytes 7.94 Gbits/sec c:\iperf>iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 208 KByte (default) ------------------------------------------------------------ [ ID] Interval Transfer Bandwidth [ 4] local 192.168.30.101 port 5001 connected with 192.168.20.122 port 25230 [ 4] 0.0-10.0 sec 10.3 GBytes 8.86 Gbits/sec
It appears to be some headroom due to virtualization but still decent numbers. This also suggest that something is happening on the FN box, so I reboot the box into Ubuntu live CD (192.168.20.110)
Code:
c:\iperf>iperf -c 192.168.20.110 ------------------------------------------------------------ Client connecting to 192.168.20.110, TCP port 5001 TCP window size: 208 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.30.101 port 52428 connected with 192.168.20.110 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 9.75 GBytes 8.38 Gbits/sec c:\iperf>iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 208 KByte (default) ------------------------------------------------------------ [ ID] Interval Transfer Bandwidth [ 4] local 192.168.30.101 port 5001 connected with 192.168.20.110 port 57248 [ 4] 0.0-10.0 sec 11.0 GBytes 9.42 Gbits/sec
The problem goes away. Was there any issue on FreeNAS with X540? As far as I know there used to be some problems with ixg drivers but that should be resolved AFAIK.