List of supported 10gb NICs in SCALE?

[jimp]

I'm seeing nice used options for the Intel X540-T1 10g and X540-T2 (two 10g ports) for roughly the same price. But while I was trying to find out if the -T2 uses the same power as the -T1 when only one port used (that's all I'll use), I found some other options which claim to be supported in Linux. However, that could mean compiling drivers, which I understand is not possible for TrueNAS Scale.

These forums generally recommend Intel. But I'm currently using a spare ASUS ROG AREION 10G NIC (Aquantia AQC107 controller), which somewhat worked in CORE, but I haven't had any issues with it in Scale. It just worked using the included aquantia.ko driver. I am considering the ASUS PEB-10G/57811-1S (1 or 2 port) to add SR-IOV support, but it is based on the Broadcom BCM 57840S or BCM 57811. I see two "Broadcom" drivers in Scale, but how do I know they support that controller?

Beyond the conventional wisdom of "just use Intel," is there a list of supported NICs (supported by the kernel, not necessarily recommended/guaranteed), or is there a way to ask my running TrueNAS Scale Linux kernel to output all supported NICs? I wouldn't mind trying alternative options, but I really don't want to waste time and money on something there isn't even an included driver for.

 

[danb35]

"Recommended" would be Intel, Chelsio, and Solarflare. Other than that, "supported" would likely include anything Debian 11 supports (as that is the basis for SCALE).

 

[jimp]

"Recommended" would be Intel, Chelsio, and Solarflare. Other than that, "supported" would likely include anything Debian 11 supports (as that is the basis for SCALE).

Thanks. I found this list: https://wiki.debian.org/DeviceDatabase/PCI

Some 10gb cards are listed, but I don't see the Aquantia AQC107 that I'm already using. The Debian list doesn't have support for any of the cards I mentioned except the Intel ones.

I never realized there isn't an official list of what the stock kernel can support. Maybe TrueNAS includes some 3rd party support, like the Aquantia driver (it worked OOTB), and I just need to buy and try. But it sure is difficult to offer testing feedback if I cannot figure out what has any chance of working with the included kernel modules, though.

 

[Arwen]

This is a problem with Linux, what it can support is questionable. And also varies based on kernel version.

I wish peripheral vendors that claimed Linux support, would list the specific kernel that started support. Even then, that does not help with RedHat Enterprise Linux, which back ports things to older kernels.


But, back to to your question, the list you found needs to be filtered based on the release used in TrueNAS SCALE. That web page does not seem to list the Debian release the device was first supported.

 

[jimp]

The closest I'm getting to figuring out what works natively without compiling/installing a driver is by looking at manufacturer provided Linux driver instructions, seeing the name of the driver, and checking if the TrueNAS Scale kernel includes it under /lib/modules/5.15.79+truenas/kernel/drivers/net.

For example, I just found a LR-LINK product that uses the Broadcom BCM57840S controller. LR-LINK provides Linux driver instructions, and source code that can be compiled on kernels 2.6+. The resulting driver is bnx2x, which the TrueNAS Scale 5.15 Linux kernel already provides. I see threads out there suggesting Broadcom cards sometimes require firmware as well, but these driver instructions make no mention of that. So I think it is likely cards based on that controller will work.

I'm starting to think the most common 10gb controllers have already made their way into the mainline Linux kernel at this point, with TrueNAS Scale providing a relatively recent kernel so they will likely work OOTB. The main reason I opened this thread is because most of my searches leading to this forum were discussions about TrueNAS Core based on FreeBSD, with advice against anything but Intel and Chelsio, but my guess is Linux support is actually much better for these less common/tested options.

I suppose even if the cards work in Linux, it could be possible TrueNAS Scale won't configure options for them correctly, like VF devices for SR-IOV and those would have to be configured separately, assuming it can even do that for Intel NICs. As long as there isn't a list of supported NICs for TrueNAS Scale specifically, this is still a buy and try approach.

 

[danb35]

but my guess is Linux support is actually much better for these less common/tested options.

I'd say that's a safe guess--hardware support under Linux generally is better than under FreeBSD. But "supported" doesn't mean it's a good idea. Realtek chips, for example, are "supported"--but they're still garbage hardware with generally garbage drivers. I don't know how Broadcom fits in; the integrated Broadcom NIC on my parents' MicroServer Gen8 seems to be working just fine, but the plural of anecdote is not data.

 

[jimp]

I'd say that's a safe guess--hardware support under Linux generally is better than under FreeBSD. But "supported" doesn't mean it's a good idea. Realtek chips, for example, are "supported"--but they're still garbage hardware with generally garbage drivers. I don't know how Broadcom fits in; the integrated Broadcom NIC on my parents' MicroServer Gen8 seems to be working just fine, but the plural of anecdote is not data.

Right. I'm not considering Realtek (I haven't even seen a 10gb option from them), but seeing Broadcom making 40gb and 100gb options gives a small vote of confidence their products are also well suited for server use. However, the main reason I'm even considering something other than Intel is the X540-T2 is really old and the X550-T2 (and certainly the X710) is out of my price range for this build.

 

[danb35]

I'd probably feel better about Mellanox than Broadcom, but I don't know that I have any real data to support that preference.

 

[jgreco]

For example, I just found a LR-LINK product that uses the Broadcom BCM57840S controller.

Please do not guess. Proceed over to the 10 Gig Networking Primer where this stuff has been summarized for you. What "works" is not the same as what works well. The Broadcom stuff is a crapfest (I write as I sit at a vendor event for a Broadcom owned company).

Mellanox than Broadcom,

Mellanox is the 10 gig world's Realtek but Broadcom is a pile of poo.

 

[JeffNAS]

That primer hasn't been updated since Apr 26, 2017 . Where can we find an official list of NICS support in CORE and SCALE?

 

[danb35]

That primer hasn't been updated since Apr 26, 2017 .

And what do you believe to be the relevance of that fact?

Where can we find an official list of NICS support in CORE and SCALE?

"Supported"? Look at the HCLs for FreeBSD 13.1 and Debian 12, respectively. "Recommended"? I don't think there have been any changes in that regard since the time of that primer that you want to dismiss. It's not like there's much development of 10 GbE hardware these days.

 

[jgreco]

That primer hasn't been updated since Apr 26, 2017 . Where can we find an official list of NICS support in CORE and SCALE?

It's still the same. There aren't a crapton of new 10 gig network cards coming out, and the ones that are, they're basically total crap like the Aquantia.

 

[jgreco]

primer hasn't been updated since Apr 26, 2017

And maybe you could explain to me why that is even relevant. Do you think good information just dies over time? Becomes stale? It's not the first time I've heard such complaints. The virtualization how-to is about ten years old but is still completely relevant; virtualization hasn't changed much and the strategies and issues I mention are still very sound and time-tested. Do I need to go around and "touch(1)" all my stickies and resources just so that they have a date of this year so that people will more readily believe them? Curious minds seriously want to understand this phenomenon.

Sep 13, 1999!