Decreased network performance after hardware upgrade

[Proxipupuce]

Hello,
First of all, I would like to apologize if my question has already been asked, I am new to the forum and even searching, I did not find anything.
Anyway, here is my problem: I recently changed the hardware of my NAS, and since the network quality has deteriorated.

To update my hardware, I turned off the server, replaced the components (motherboard, processor and RAM), then turned it back on.

I first had a system time problem, indeed the reports were arriving two hours late. I solved this problem by changing the time zone twice. However, I lost all the reports of the old configuration.

My current problem is different: with the old configuration, my local download speed was up to 11MB/s and remained stable, while with the new configuration it is 8MB/s and not stable. I want to specify that my low download speed is due to the fact that I use a PLC to connect my NAS.

Old configuration :
- Motherboard : Asus P8H61-MX
- Processor : Intel I3-2130
- RAM : Crucial 2*4gb 1333Mhz
- System installed on a SSD connected in SATA 3Gb/s
- 3 western digital red of 3 to in RAIDZ1 connected in SATA 3Gb/s

New configuration :
- Motherboard : MSI B75MA-E33
- Processor: Intel I5-3570
- RAM: Viper 2*4gb 1600Mhz
- System installed on a SSD connected in SATA 6Gb/s
- 3 western digital red of 3 to in RAIDZ1 connected in SATA 3Gb/s

What I don't understand is why the network degraded when both motherboards use the same network controller, a Realtek 8111E.

Don't hesitate to ask me for more information, thanks in advance to anyone who can help me!

 

[jgreco]

What I don't understand is why the network degraded when both motherboards use the same network controller, a Realtek 8111E.

With only 8GB of RAM, you may not be able to cache useful data in the ARC. Performance with minimal RAM is always a little weird.

Further, the 8111E is a horrible ethernet, please see the resource:

Is my Realtek ethernet really that bad?

One of the longest running issues faced here in the forums may be people who come to these forums, wanting to recycle an old PC into a NAS. In general, this is a bad idea because typical PC hardware isn't optimized towards the task. However...

www.truenas.com

It's also possible that the 8111E chip on the MSI board is a knockoff chip. Due to the pressures to sell PC gear for the minimum possible cost, it sometimes happens that parts are acquired in a back alley over in Shenzhen from an illegitimate manufacturer. Buying ten thousand ethernet chips off some knockoff production line might reduce the cost to manufacture the board a bit, and sometimes knockoff chips are not fully functioning chips. Remember that in a lot of applications, Windows users won't really know or care if their ethernet isn't quite up to snuff.

 

[Proxipupuce]

Thanks for your answer!

As for the 8GB of RAM, I'm already planning to change this in the next few weeks.

Tired of my PLC, I connected my NAS to the MESH Orbi system in my house yesterday. With this system, I can get my computer up to 50MB/S on the internet connection, and much more on my local network.

However, on my NAS, in internet connection and local, I never exceed 20MB/S.

I checked everything, all the devices are connected in 1GB/S, with cat6 cables.

So I'm going to do what you recommended, and buy a PCIe Ethernet card, I would also disable the controller embedded in the motherboard.
On the post you sent me, it is recommended to buy an Intel Desktop CT. Do you have any others to advise me? A 1GB/S card is sufficient in my case.

Thanks in advance !

 

[Whattteva]

Have you done an iperf test to see if it's actually your ethernet bottlenecking and not the disks maybe?

 

[jgreco]

it is recommended to buy an Intel Desktop CT. Do you have any others to advise me?

Not really. The upside to the Intel 1G controller is that it has an Intel-authored driver and that they put years of work into perfecting it with multiple chipsets. It became more or less the industry standard for servers; AMD based systems sometimes went with Broadcom, but there are sometimes problems with those, so I don't recommend them. You don't need the more expensive "fancy" Intel server PCIe cards because the fancy features that become available on those cards simply don't make much difference.

 

[Proxipupuce]

Have you done an iperf test to see if it's actually your ethernet bottlenecking and not the disks maybe?

I tried, but couldn't figure out how to do it.
On the other hand, I ran the command "diskinfo -ctv /dev/ada" on all my HDDs, with a speed always above 120-130MB/s (500 on my SSD).

 

[Whattteva]

I tried, but couldn't figure out how to do it.
On the other hand, I ran the command "diskinfo -ctv /dev/ada" on all my HDDs, with a speed always above 120-130MB/s (500 on my SSD).

You need two computers with iperf installed to do this. Note that iperf3 is preinstalled on both TrueNAS CORE and SCALE.

For this example, I will use iperf3:
On one end, you do iperf3 -s.
On the other end, you do iperf3 -c <ip_address_of_other_computer>

Here's an example of how it might look like:

Code:

root@pve1:~# iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 172.31.0.16, port 47826
[  5] local 172.16.3.1 port 5201 connected to 172.31.0.16 port 47834
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.45 GBytes  21.0 Gbits/sec               
[  5]   1.00-2.00   sec  2.44 GBytes  21.0 Gbits/sec               
[  5]   2.00-3.00   sec  2.59 GBytes  22.2 Gbits/sec               
[  5]   3.00-4.00   sec  2.59 GBytes  22.3 Gbits/sec               
[  5]   4.00-5.00   sec  2.66 GBytes  22.8 Gbits/sec               
[  5]   5.00-6.00   sec  2.59 GBytes  22.3 Gbits/sec               
[  5]   6.00-7.00   sec  2.54 GBytes  21.8 Gbits/sec               
[  5]   7.00-8.00   sec  2.59 GBytes  22.3 Gbits/sec               
[  5]   8.00-9.00   sec  2.62 GBytes  22.5 Gbits/sec               
[  5]   9.00-10.00  sec  2.61 GBytes  22.4 Gbits/sec               
[  5]  10.00-10.04  sec   119 MBytes  23.0 Gbits/sec               
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.04  sec  25.8 GBytes  22.1 Gbits/sec                  receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

Code:

root@debian2:~# iperf3 -c 172.16.3.1
Connecting to host 172.16.3.1, port 5201
[  5] local 172.31.0.16 port 47834 connected to 172.16.3.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  2.55 GBytes  21.9 Gbits/sec    0   3.02 MBytes     
[  5]   1.00-2.00   sec  2.45 GBytes  21.0 Gbits/sec    0   3.02 MBytes     
[  5]   2.00-3.00   sec  2.58 GBytes  22.1 Gbits/sec    0   3.02 MBytes     
[  5]   3.00-4.00   sec  2.61 GBytes  22.4 Gbits/sec    0   3.02 MBytes     
[  5]   4.00-5.00   sec  2.66 GBytes  22.9 Gbits/sec    0   3.02 MBytes     
[  5]   5.00-6.00   sec  2.58 GBytes  22.2 Gbits/sec    0   3.02 MBytes     
[  5]   6.00-7.00   sec  2.54 GBytes  21.8 Gbits/sec    0   3.02 MBytes     
[  5]   7.00-8.00   sec  2.60 GBytes  22.3 Gbits/sec    0   3.02 MBytes     
[  5]   8.00-9.00   sec  2.61 GBytes  22.5 Gbits/sec    0   3.02 MBytes     
[  5]   9.00-10.00  sec  2.62 GBytes  22.5 Gbits/sec    0   3.02 MBytes     
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  25.8 GBytes  22.2 Gbits/sec    0             sender
[  5]   0.00-10.04  sec  25.8 GBytes  22.1 Gbits/sec                  receiver

iperf Done.

 

[Proxipupuce]

You need two computers with iperf installed to do this.

For this example, I will use iperf3:
On one end, you do iperf3 -s.
On the other end, you do iperf3 -c <ip_address_of_other_computer>

and that's it really.

I would install Iperf on my computer and do this, I will post the results shortly!

 

[Proxipupuce]

Not really. The upside to the Intel 1G controller is that it has an Intel-authored driver and that they put years of work into perfecting it with multiple chipsets. It became more or less the industry standard for servers; AMD based systems sometimes went with Broadcom, but there are sometimes problems with those, so I don't recommend them. You don't need the more expensive "fancy" Intel server PCIe cards because the fancy features that become available on those cards simply don't make much difference.

Okay, thanks.

For the drivers, I remember reading something about TRUENAS natively supporting these cards, is it true? So there is no need to install the drivers?

Also, the card mentioned above doesn't seem to be available for purchase for a reasonable price anymore.

Is a card with an intel I225-V chip, like the ones available on the links below, reliable and without knockoff chips?

First card : https://www.amazon.fr/gp/product/B096M73LGT/ref=ox_sc_act_title_1?smid=A32MB0CVDT9W8H&psc=1

Second card : https://www.amazon.fr/gp/product/B0BNHWZBCC/ref=ox_sc_act_title_1?smid=A2URVFJGMWKSOV&psc=1

If these cards are not good, do you have links or references of real intel cards please?

Thanks in advance !

 

[Proxipupuce]

You need two computers with iperf installed to do this. Note that iperf3 is preinstalled on both TrueNAS CORE and SCALE.

[ ID] Interval Transfer Bitrate
[ 5] 0.00-1.00 sec 8.78 MBytes 73.7 Mbits/sec
[ 5] 1.00-2.00 sec 9.22 MBytes 77.3 Mbits/sec
[ 5] 2.00-3.00 sec 9.56 MBytes 80.2 Mbits/sec
[ 5] 3.00-4.01 sec 9.44 MBytes 78.7 Mbits/sec
[ 5] 4.01-5.00 sec 8.20 MBytes 69.2 Mbits/sec
[ 5] 5.00-6.00 sec 9.49 MBytes 79.6 Mbits/sec
[ 5] 6.00-7.00 sec 8.89 MBytes 74.6 Mbits/sec
[ 5] 7.00-8.00 sec 9.46 MBytes 79.4 Mbits/sec
[ 5] 8.00-9.00 sec 9.96 MBytes 83.5 Mbits/sec
[ 5] 9.00-10.00 sec 11.0 MBytes 92.0 Mbits/sec
[ 5] 10.00-10.01 sec 139 KBytes 77.2 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate
[ 5] 0.00-10.01 sec 94.1 MBytes 78.8 Mbits/sec receiver

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.02 sec 9.00 MBytes 74.4 Mbits/sec
[ 4] 1.02-2.00 sec 9.25 MBytes 78.6 Mbits/sec
[ 4] 2.00-3.00 sec 9.50 MBytes 79.8 Mbits/sec
[ 4] 3.00-4.00 sec 9.25 MBytes 77.5 Mbits/sec
[ 4] 4.00-5.01 sec 8.38 MBytes 70.0 Mbits/sec
[ 4] 5.01-6.00 sec 9.38 MBytes 78.8 Mbits/sec
[ 4] 6.00-7.00 sec 9.00 MBytes 75.5 Mbits/sec
[ 4] 7.00-8.00 sec 9.50 MBytes 79.6 Mbits/sec
[ 4] 8.00-9.00 sec 10.0 MBytes 84.1 Mbits/sec
[ 4] 9.00-10.00 sec 10.9 MBytes 91.2 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 94.1 MBytes 78.9 Mbits/sec sender
[ 4] 0.00-10.00 sec 94.1 MBytes 78.9 Mbits/sec receiver

iperf Done.

after doing a test, I got these results.

 

[Whattteva]

after doing a test, I got these results.

Those speeds are awful for a 1G ethernet. You should be getting at the very least 700Mbps. Your ethernet is clearly bottlenecking you. Those speeds look more like 100M Fast Ethernet speeds.

You might want to check if you're actually connected at 1G link speed rather than 100M.

 

[Proxipupuce]

Those speeds are awful for a 1G ethernet. You should be getting at the very least 700Mbps. Your ethernet is clearly bottlenecking you.

You might want to check if you're actually connected at 1G or 100M (Fast Ethernet).

And apparently it's coming from the realtek chip in my NAS... I've already started looking for a genuine Intel PCIe network card.

Do not hesitate if you have references to advise me, and thank you for your help.

 

[Whattteva]

And apparently it's coming from the realtek chip in my NAS... I've already started looking for a genuine Intel PCIe network card.

Do not hesitate if you have references to advise me, and thank you for your help.

Forgot to mention that you should also try to swap the cables and see if that will help speeds. Just a couple days ago, I was getting similar issue as you (PC connecting only at 100M, which results in 77Mbps iperf). I swapped cables and immediately, the speeds went up to 700+ Mbps.

 

[Proxipupuce]

Ces vitesses sont terribles pour un Ethernet 1G. Vous devriez obtenir au moins 700 Mbps. Votre Ethernet vous bloque clairement. Ces vitesses ressemblent plus à des vitesses Fast Ethernet de 100M.

Vous voudrez peut-être vérifier si vous êtes réellement connecté à une vitesse de liaison de 1G plutôt qu'à 100M.

Like this, with " ifconfig | grep media" ?

 

[Whattteva]

Like this, with " ifconfig | grep media" ?

Yes, but it probably helps to see which one of those "media" lines belong to the actual NIC you're using.

 

[Proxipupuce]

Yes, but it probably helps to see which one of those "media" lines belong to the actual NIC you're using.

How to do it ?

Also, I did a new test.

I plugged in a laptop instead of my server and tested again.
On internet, I reach 350mb/s, but, with iperf, impossible to exceed 75mb/s.

Same thing on the server:
On internet, I reach 180mb/s.
When transferring video files from the server to the PC I also get 180mb/s.
But with iperf, impossible to exceed 75mb/s.

I don't think my pc is a problem, because on it I easily exceed 400mb/s...
Et si le serveur était à 100 Mo/s, il ne pourrait pas atteindre 180 Mo/s en téléchargement, n'est-ce pas ?

Moreover, all cables are Cat6.

 

[Proxipupuce]

I tried to change the server cable by another cat 8 cable, still the same problems...
200mb/s on internet and video transfer, but 75-80mb/s in iperf...

 

[Whattteva]

I tried to change the server cable by another cat 8 cable, still the same problems...
200mb/s on internet and video transfer, but 75-80mb/s in iperf...

By "internet" how exactly are you performing this test? Trust me, iperf is probably more accurate than whatever it is you're using, especially if you're using a browser repeatedly on the same site because it may cache stuff behind the scenes.

 

[Proxipupuce]

By "internet" how exactly are you performing this test? Trust me, iperf is probably more accurate than whatever it is you're using, especially if you're using a browser repeatedly on the same site because it may cache stuff behind the scenes.

On the server, I use the OpenSpeedTest plugin.
On the laptop, I use the speedtest by Ookla.
For the video transfer from the server to my desktop, I look at the file transfer speed.

 

[Proxipupuce]

I also tried an iperf between my desktop and my laptop, still the same result, 80mb/s, yet both are connected in 1Gb/s, I checked.