Slow Ethernet speed

[ilkor]

Hi i have a very strange problem with the speed of the Ethernet connection with my Thecus W4000 Freenas server.
Tests with iperf3 are made between NAS device and the Ubuntu machine directly connected in a switch. Network runs at 1Gb. The cables and the switch are checked - they are not relevant to the problem. Test results are very strange and vary depending on which machine the iperf3 server is running on.
FreeNAS server - Ubuntu client = 146Mb/s
FreeNAS client - Ubuntu server = 515Mb/s
Originally, I bought this device with Windows Server 2012 and had no such issues until I installed FreeNAS. The only modification I've done is increasing the RAM from 4GB to 8GB. More information about the W4000 hardware
I searched the forum and saw that this hardware was used by others. Does anyone know how this can be overcome?

I apply the test logs in both directions:

---------------------------------------------------------------------------------- ilkor@homeserver:~$ iperf3 -c 10.0.0.4 Connecting to host 10.0.0.4, port 5201 [ 4] local 10.0.0.2 port 56992 connected to 10.0.0.4 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 18.7 MBytes 156 Mbits/sec 36 129 KBytes [ 4] 1.00-2.00 sec 18.8 MBytes 158 Mbits/sec 35 129 KBytes [ 4] 2.00-3.00 sec 16.7 MBytes 140 Mbits/sec 26 129 KBytes [ 4] 3.00-4.00 sec 17.4 MBytes 146 Mbits/sec 26 129 KBytes [ 4] 4.00-5.00 sec 19.0 MBytes 159 Mbits/sec 18 129 KBytes [ 4] 5.00-6.00 sec 18.9 MBytes 158 Mbits/sec 27 129 KBytes [ 4] 6.00-7.00 sec 17.3 MBytes 145 Mbits/sec 22 129 KBytes [ 4] 7.00-8.00 sec 14.4 MBytes 121 Mbits/sec 18 129 KBytes [ 4] 8.00-9.00 sec 16.7 MBytes 140 Mbits/sec 6 129 KBytes [ 4] 9.00-10.00 sec 16.8 MBytes 141 Mbits/sec 12 129 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 175 MBytes 146 Mbits/sec 226 sender [ 4] 0.00-10.00 sec 174 MBytes 146 Mbits/sec receiver iperf Done. ---------------------------------------------------------------------------------- root@NAS[~]# iperf3 -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 10.0.0.2, port 56990 [ 5] local 10.0.0.4 port 5201 connected to 10.0.0.2 port 56992 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 17.9 MBytes 150 Mbits/sec [ 5] 1.00-2.00 sec 18.8 MBytes 158 Mbits/sec [ 5] 2.00-3.00 sec 16.7 MBytes 140 Mbits/sec [ 5] 3.00-4.00 sec 17.4 MBytes 146 Mbits/sec [ 5] 4.00-5.00 sec 19.0 MBytes 159 Mbits/sec [ 5] 5.00-6.00 sec 18.9 MBytes 158 Mbits/sec [ 5] 6.00-7.00 sec 17.2 MBytes 145 Mbits/sec [ 5] 7.00-8.00 sec 14.5 MBytes 121 Mbits/sec [ 5] 8.00-9.00 sec 16.6 MBytes 140 Mbits/sec [ 5] 9.00-10.00 sec 16.8 MBytes 141 Mbits/sec [ 5] 10.00-10.01 sec 116 KBytes 133 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate [ 5] 0.00-10.01 sec 174 MBytes 146 Mbits/sec receiver ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- ^Ciperf3: interrupt - the server has terminated -------------------------------------------------------------------------------

---------------------------------------------------------------------------------- root@NAS[~]# iperf3 -c 10.0.0.2 Connecting to host 10.0.0.2, port 5201 [ 5] local 10.0.0.4 port 59624 connected to 10.0.0.2 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 36.6 MBytes 307 Mbits/sec 0 3.00 MBytes [ 5] 1.00-2.00 sec 46.0 MBytes 386 Mbits/sec 0 3.00 MBytes [ 5] 2.00-3.00 sec 59.2 MBytes 497 Mbits/sec 0 3.00 MBytes [ 5] 3.00-4.00 sec 52.3 MBytes 439 Mbits/sec 0 3.00 MBytes [ 5] 4.00-5.00 sec 67.3 MBytes 564 Mbits/sec 0 3.00 MBytes [ 5] 5.00-6.00 sec 87.1 MBytes 732 Mbits/sec 0 3.00 MBytes [ 5] 6.00-7.00 sec 62.7 MBytes 526 Mbits/sec 0 3.00 MBytes [ 5] 7.00-8.00 sec 80.9 MBytes 678 Mbits/sec 0 3.00 MBytes [ 5] 8.00-9.00 sec 59.8 MBytes 502 Mbits/sec 0 3.00 MBytes [ 5] 9.00-10.00 sec 61.8 MBytes 517 Mbits/sec 0 3.00 MBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 614 MBytes 515 Mbits/sec 0 sender [ 5] 0.00-10.00 sec 614 MBytes 515 Mbits/sec receiver iperf Done. ---------------------------------------------------------------------------------- ilkor@homeserver:~$ iperf3 -s ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- Accepted connection from 10.0.0.4, port 17004 [ 5] local 10.0.0.2 port 5201 connected to 10.0.0.4 port 59624 [ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 2.92 MBytes 24.5 Mbits/sec [ 5] 1.00-2.00 sec 41.3 MBytes 346 Mbits/sec [ 5] 2.00-3.00 sec 46.5 MBytes 390 Mbits/sec [ 5] 3.00-4.00 sec 62.9 MBytes 528 Mbits/sec [ 5] 4.00-5.00 sec 57.1 MBytes 479 Mbits/sec [ 5] 5.00-6.00 sec 62.6 MBytes 525 Mbits/sec [ 5] 6.00-7.00 sec 86.3 MBytes 724 Mbits/sec [ 5] 7.00-8.00 sec 60.5 MBytes 507 Mbits/sec [ 5] 8.00-9.00 sec 80.1 MBytes 672 Mbits/sec [ 5] 9.00-10.00 sec 67.9 MBytes 570 Mbits/sec [ 5] 10.00-10.83 sec 45.5 MBytes 462 Mbits/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 5] 0.00-10.83 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.83 sec 614 MBytes 476 Mbits/sec receiver ----------------------------------------------------------- Server listening on 5201 ----------------------------------------------------------- ^Ciperf3: interrupt - the server has terminated ----------------------------------------------------------------------------------

 

[ilkor]

Solution:
Since I did not find a software solution to this problem I was able to work around it by adding a third 1000 Mbps network adapter into the PCIe slot of Thecus W4000.

Code:

root@NAS[~]# iperf3 -c 10.0.0.2
Connecting to host 10.0.0.2, port 5201
[  5] local 10.0.0.4 port 39013 connected to 10.0.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   113 MBytes   945 Mbits/sec    0    262 KBytes
[  5]   1.00-2.00   sec   112 MBytes   937 Mbits/sec    0    328 KBytes
[  5]   2.00-3.00   sec   112 MBytes   942 Mbits/sec    0    379 KBytes
[  5]   3.00-4.00   sec   112 MBytes   942 Mbits/sec    0    429 KBytes
[  5]   4.00-5.00   sec   112 MBytes   942 Mbits/sec    0    472 KBytes
[  5]   5.00-6.00   sec   112 MBytes   942 Mbits/sec    0    506 KBytes
[  5]   6.00-7.00   sec   112 MBytes   942 Mbits/sec    0    539 KBytes
[  5]   7.00-8.00   sec   112 MBytes   942 Mbits/sec    0    572 KBytes
[  5]   8.00-9.00   sec   112 MBytes   942 Mbits/sec    0    607 KBytes
[  5]   9.00-10.00  sec   112 MBytes   942 Mbits/sec    0    637 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.10 GBytes   942 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  1.10 GBytes   941 Mbits/sec                  receiver

 

[Ed Clarke]

I also was getting unexpectedly slow speeds between my Win/10 box and a FreeNAS 11.3 server. The server's a Dell R410 with a 10gb fiber link to a Quanta LB4M switch. The Windows box has a 1GB link over an unmarked ethernet cable to the same switch. Throughput was roughly 500 megabits/sec on several tests.

After looking through Youtube for iperf issues, I came across a corny video where the guy was testing iperf with wireless laptops to a switch. He had crap throughput, couldn't change the laptop wireless cards and had an "AH HA!" thought - his cables were CAT5 between the access point and his target. A switch to CAT6 cured his problem. Now this sounded like complete BS to me but I had a newer CAT6 cable on the shelf next to me and tried it. 930 megabits/second. WTF??? I thought this CAT5 vs CAT6 was all marketing hype but it looks like it's not. Both the switch and the Windows box were locked at 1GB/sec full duplex for both sets of tests.

I can test CAT5 cables with my Fluke DSP-100 but there's no way that I can test CAT6 or above. What do others use to test these kinds of cables ( and that won't cost more than my servers and clients combined )?

 

[jgreco]

Don't fall for the Monster Cable argument.

It's all digital. It is absolutely possible to have a bad cable. It's absolutely possible to have a bad Cat6 cable too. But these things will show, you'll see errors and problems reported in the ethernet statistics at each end.

For the most part, a short Cat5 cable will perform about the same as a short Cat6. The problems begin to crop up when you're going longer, especially if the Cat5 cable signal quality is closer to the low end of the MHz range.

For basic testing, use a Klein VDV Scout 2. They're cheap and they identify common miswiring issues, which happen far more often.

I keep a Platinum Tools Net Prowler Pro in my main cabling bag. At about ten times the cost of a Scout 2, it does a lot more, but on average is only maybe twice as valuable to me as a Scout 2. The next step up are certification tools that cost crazy sums. I prefer to monitor for interface errors.

 

[Ed Clarke]

Well, it turns out to be "user error". I very carefully tested the "in the wall" cabling that I installed using CAT6 cable. I never tested the patch cords that go from the wall to the computer or to the switches. At least one of my patch cables had hideous NeXT (near-end cross talk) even though they "worked" according to a continuity tester. I suspect that there are more...

Moral of the story is - If you have good test equipment and don't use it then you deserve what you get ( less than half what I was paying for from Verizon ). Now I have a big pile of ethernet cables to go through to see if they're worth keeping. I'm going to keep with the DSP-100/DSP-SR combination and USE IT. Time to go do some testing while hiding my head in shame.