2.5gbe network card for SCALE: Realtek or Intel drivers?

[Davvo]

Are you able to test it with CORE as well? Unlikely ti work, but who knows...

 

[Patrick_3000]

Are you able to test it with CORE as well? Unlikely ti work, but who knows...

I have no ability to do that. First, because I don't currently have CORE installed on anything and, second, because having OPNsense, the firewall and router for my house, running as a VM on SCALE with the QNAP card connecting to the modem means I don't plan to shut down or even reboot more than once or twice a year.

 

[Etorix]

Are you able to test it with CORE as well? Unlikely ti work, but who knows...

Presumably, it works as well as 10 GbE Aquantia NICs of the same family—that is, not reliably enough for some tastes.

 

[ChrisRJ]

It all depends on your definition of "it works": In a NAS context for me that means to be as reliable as possible and not give up. That is considerably different from "I got it to connect and transmit some test data".

 

[Patrick_3000]

I don't agree with the sentiment that Intel is the only thing worth considering. Personally, I've had numerous NICs over the years, and I've had a couple of problems with Intel NICs. The worst problem I ever had was the 2.5 gb Intel NIC when I was running OPNsense bare metal that connected to the modem. Like others have reported, when it dropped due to an internet outage, it didn't come back online unless OPNsense was rebooted. This is reportedly due to power saving features. I also had an old Intel quad port 1gb NIC years ago that started connecting at fast ethernet (100 mbps) speeds only.

I also have an intermittent problem currently with a 10 gb Realtek NIC in a Windows PC that occasionally drops after reboots, which happen frequently with Windows.

I've never had a problem with Aquantia NICs, but then again, I've owned far fewer of them than Intel or Realtek NICs. In fact, I've only owned two that I can remember. So, I don't have much data on them.

And sure, that's all anecdotal, but so is everything I'm hearing in this thread and similar threads, including by "Intel only" advocates.

If anyone knows of quantitative data showing that Intel, Aquantia, or even Realtek is any better or worse than the others, I would find that extremely helpful. And by quantititave data, I mean a study of hundreds or thousands of deployed NICs with different chipsets and statistical data on uptime and downtime.

 

[Etorix]

You're misreading the room. There's no "Intel only" policy. Chelsio and Solarflare NICs are also recommended—Chelsio possible even ahead of Intel. But no 2.5 GbE NIC is recommended—not even those from Intel.

10 Gig Networking Primer

This is a discussion of high speed networking for newcomers, with specific emphasis on practical small- to medium-scale deployments for home labs or small office users. It originated with a forum thread located here that has many pages of...

www.truenas.com

 

[Patrick_3000]

You're misreading the room. There's no "Intel only" policy. Chelsio and Solarflare NICs are also recommended—Chelsio possible even ahead of Intel. But no 2.5 GbE NIC is recommended—not even those from Intel.

10 Gig Networking Primer

This is a discussion of high speed networking for newcomers, with specific emphasis on practical small- to medium-scale deployments for home labs or small office users. It originated with a forum thread located here that has many pages of...

www.truenas.com

As I've mentioned earlier in this thread, I'm no advocate for 2.5 gbe. I prefer 10 gbe and would rather use it. However, the only cable modems faster than 1 gbps are 2.5 gbps. In particular, there are no 10 gbps cable modems made by any manufacturer, at least based on my research. So, if you're using a NIC to connect to a cable modem, and your internet feed is faster than 1 gbps, as mine is (it's 1.25 gbps), then in order to take full advantage of the speed you're paying for, you absolutely need a NIC capable of connecting at 2.5 gbe.

Given this, you have limited options. I doubt, for example, that Chelsio and Solarflare, which cater to the server and datacenter world, make any NIC cabable of connecting at NBase-T speeds such as 2.5 gbps. Probably it's just 1, 10, and higher gbps.

So, you're looking at Intel, Realtek, or Aquantia. Intel 2.5 gb cards have documented problems with drops that I personally have experienced. I will never return to using an Intel 2.5 gb card. As for Realtek, people on here tend to be anti-Realtek, and as I noted, even I have experienced problems with a 10 gb Realtek card in a Windows PC that occasionally drops after rebooting.

That pretty much leaves two realistic options that I'm aware of for connecting to a 2.5 gb modem: an Intel 10 gb card that can negotiate down to 2.5 gb, such as the x550-T2, or Aquantia. Both, in my opinion, are reasonable options.

 

[Etorix]

In particular, there are no 10 gbps cable modems made by any manufacturer, at least based on my research.

Here are two:

5302 XGS-PON ONT - EPSGlobal

Cost optimized 10 Gigabit X-Series ONT enables subscribers to enjoy bandwidth intensive multimedia services with leading-edge XGS-PON technology.

www.epsglobal.com

TIM Sagemcom 10 Gb

Schede tecniche e guide all'installazione del nuovo modem TIM Sagemcom 10 Gb: configura e collega tutti i dispositivi per la tua attività o della tua abitazione.

www.tim.it

The second is the one used by Telecom Italia (TIM) to serve 10 Gb/s fibre to end users.

TIM Super 10 Gigabit in 10 città: costi e dettagli della sperimentazione XGS-PON Fibra FTTH

Dal 1° Ottobre 2021 TIM ha incrementato la copertura dell'offerta sperimentale XGS-PON in Fibra FTTH, TIM Super 10 Gigabit, adesso in 10 città

www.mondomobileweb.it

There are experiments (or maybe even deployments by now?) in France, Spain and the Netherlands. Sorry you're on the wrong continent…

So, if you're using a NIC to connect to a cable modem, and your internet feed is faster than 1 gbps, as mine is (it's 1.25 gbps), then in order to take full advantage of the speed you're paying for, you absolutely need a NIC capable of connecting at 2.5 gbe.

Why do you want to connect your NAS directly to the Internet provider box? You connect the box to a suitable switch, and operate your local network at 10 Gb/s for the most capable desktops and the NAS, 1 or 2.5 Gb/s for other clients and 2.5 Gb/s link to outside.

QSW-M2106-4S | 10GbE and 2.5GbE Layer 2 Web Managed Switch for SMB Network Deployment

The QSW-M2106-4S is a Layer 2 Web Managed Switch with four 10GbE SFP+ fiber ports and six 2.5 Gigabit ports. Supporting Layer 2 switching and network management via a user-friendly web user interface, the QSW-M2106-4S offers flexible deployment in hybrid high-speed network environments and...

www.qnap.com

 

[Patrick_3000]

Here are two:

5302 XGS-PON ONT - EPSGlobal

Cost optimized 10 Gigabit X-Series ONT enables subscribers to enjoy bandwidth intensive multimedia services with leading-edge XGS-PON technology.

www.epsglobal.com

TIM Sagemcom 10 Gb

Schede tecniche e guide all'installazione del nuovo modem TIM Sagemcom 10 Gb: configura e collega tutti i dispositivi per la tua attività o della tua abitazione.

www.tim.it

The second is the one used by Telecom Italia (TIM) to serve 10 Gb/s fibre to end users.

TIM Super 10 Gigabit in 10 città: costi e dettagli della sperimentazione XGS-PON Fibra FTTH

Dal 1° Ottobre 2021 TIM ha incrementato la copertura dell'offerta sperimentale XGS-PON in Fibra FTTH, TIM Super 10 Gigabit, adesso in 10 città

www.mondomobileweb.it

There are experiments (or maybe even deployments by now?) in France, Spain and the Netherlands. Sorry you're on the wrong continent…


Why do you want to connect your NAS directly to the Internet provider box? You connect the box to a suitable switch, and operate your local network at 10 Gb/s for the most capable desktops and the NAS, 1 or 2.5 Gb/s for other clients and 2.5 Gb/s link to outside.

QSW-M2106-4S | 10GbE and 2.5GbE Layer 2 Web Managed Switch for SMB Network Deployment

The QSW-M2106-4S is a Layer 2 Web Managed Switch with four 10GbE SFP+ fiber ports and six 2.5 Gigabit ports. Supporting Layer 2 switching and network management via a user-friendly web user interface, the QSW-M2106-4S offers flexible deployment in hybrid high-speed network environments and...

www.qnap.com

So, some 10gb cable modems exist in other countries that I could possibly acquire online somewhere, probably for a high cost, but that are probably useless to me because they almost certainly wouldn't work with a U.S. internet service provider. That's not a solution.

Also, you suggest connecting the cable modem to a switch. But why? How is that any more reliable or useful than connecting it to a NIC? And why all the extra expense and hassle? Is it just to avoid an Aquantia card or an Intel 10 gb card that can negotiate to 2.5 gb?

To be clear, I already have a setup that works. And my local network does operate at 10 gbps, at least a portion of it does. I have a switch that has 10 gbps and 1 gbps links. I have a NIC for connecting to the cable modem, which I believe is rather reliable--a QNAP card specifically designed for NAS that has the Aquantia chipset. Aquantia developed the N-Base T (2.5 gb) standard.

Bottom line: In the United States of America, if someone is using SCALE to host their firewall in a VM, and needs to pass the firewall a 2.5 gbe interface connected to a cable modem due to having internet faster than 1 gbe, then I highly recommend using either the QNAP QXG-5G1T-111C card (or its dual or quad counterpart) for this purpose, or else an Intel 10 gb card based on the x550-T2 chipset. Those are the only realistic, simple options that I'm aware of, and the advantage of the QNAP card over an Intel x550-T2 is that the QNAP card uses only a single PCIE 3.0 lane, which was nice in my case because I was able to put it in an x1 slot and save my larger slots for other things.

 

[rebootman]

Query for Users, but I noticed a few others with opnsense intel 2.5gb nic users mentioning they had disconnections.

I am also experiencing random network drops with my OpenwrtX86 mini router which has intel i226 chipset, I am still testing things out but does it appear the problem with 2.5gb Intel i225/6 network connection drops is still occurring with people using linux and windows platforms?

 

[stillka]

Hello,
in TrueNAS-SCALE-23.10.1.1 release there is bug in the kernel causing RTL8125 Ethernet adapters can't work with mtu 9000:

218187 – r8169: no carrier with RTL8125

bugzilla.kernel.org

So I have ordered I226 T1 2.5Gbps Ethernet adapter and I will test it with my setup, I'll post update here about results!
(boughts ASHATA 2.5Gbps PCIe Network Card, I226 T1 PCI E X1 Single Port Network Adapter, 2500/1000/100Mbps PCI Express Gigabit Ethernet Card, RJ45 LAN Controller)

 

[MrGuvernment]

Hello,
in TrueNAS-SCALE-23.10.1.1 release there is bug in the kernel causing RTL8125 Ethernet adapters can't work with mtu 9000:

218187 – r8169: no carrier with RTL8125

bugzilla.kernel.org

So I have ordered I226 T1 2.5Gbps Ethernet adapter and I will test it with my setup, I'll post update here about results!
(boughts ASHATA 2.5Gbps PCIe Network Card, I226 T1 PCI E X1 Single Port Network Adapter, 2500/1000/100Mbps PCI Express Gigabit Ethernet Card, RJ45 LAN Controller)

Most times you do not actually need 9000 MTU either.

 

[HansBohne]

I know Realtek cards are not that popular - but before changing mainboard on the TrueNAS Server and upgrading to 10 GBit I used a Realtek 2.5 GBit USB 3 on the TrueNAS and transferred abot 700 TByte combined without any problems - at an average speed of ~ 260 MByte / sec (reading and writing)

I personally don't think Realtek is a really bad choice for home usage - though the manual udev patching for the usb 2.5 GBit Realtek in Linux is a bit annoying - but when it's done it works.

My AQTion 10 GBit cards are stuck Version 0.5 in FreeBSD - and there were no changes for many years - you have to compile them by hand - but they also do their job ... unless freebsd updates the kernel => kernel panic next boot until you reompile :D

 

[ChrisRJ]

I personally don't think Realtek is a really bad choice for home usage - though the manual udev patching for the usb 2.5 GBit Realtek in Linux is a bit annoying - but when it's done it works.

This is your personal experience, which seems to be quite positive and I am happy for you that it is.

But generalizing from it against the contrary experience of countless instances documented here in the forum alone (and there very likely more to be found elsewhere), is not helpful to those people that you want to help.

Also "it works" is a term to be careful with. There is a reason why this is usually formally defined in project contracts: It means different things to different people.

I am aware that all this may sound patronizing, although I can assure you it isn't meant to be. It is simply the overall experience of many people who have been doing this for living for decades.

 

[HansBohne]

Though on the other hand many billions(?) of computers have combined used Realtek chips for decades ...

A thousand reports of problems are still only ~ 0,0001%,... that would also be still "nothing" in terms of statistics - maybe so many perople have problems with Realtek because many (consumer) boards have them onboard?

Perfomance is probably a bit faster with Intel especially if you have a very low end cpu or very high loads all the time + high network traffic.

 

[MrGuvernment]

Though on the other hand many billions(?) of computers have combined used Realtek chips for decades ...

A thousand reports of problems are still only ~ 0,0001%,... that would also be still "nothing" in terms of statistics - maybe so many perople have problems with Realtek because many (consumer) boards have them onboard?

Perfomance is probably a bit faster with Intel especially if you have a very low end cpu or very high loads all the time + high network traffic.

Billions yes, which the largest majority also run Windows.. which Realtek tend to work fine in because the drivers are more mature. Realtek under BSD is a whole other story and even under Linux Realtek has always been iffy. Yes, support has gotten better over I think the last year.. but it is not where it should be.

There is a reason why you do not see Realtek NIC in enterprise level gear (storage / servers et cetera) and why Chelsio / mella and intel tend to reign supreme.

 

[M3PH]

No, Realtek sucks. Intel 225v Gen 3, or 226, are reported to be solid, at least as much as anything 2.5 GbE is.

the 225 has some firmware bugs that cause issues, epsecially when using smb to transfer data. it's better to get a 226 or a x550 (but even the 550's have issues with link autodetect and 2.5gbe and 5gbe and require you to manually select the mode).