Onboard NIC not seen after installing extra Nic

[Red_November]

Hello I was hoping someone might be able to offer some idea on why this is happening

I have a 2 port onboard Intel 82574L Gigabit network interface, When I installed a Intel 82599ES network card

The Intel 82574 interface was no longer seen when running

pciconf -lv | grep -B3 network or dmesg | grep -i ethernet scan

I checked in the BIOS and it was still enabled. Removing the addon card enabled the onboard card again. Could any one tell me why this is happening or to maybe trouble shoot to find how to enable it.

 

[jgreco]

You really need to go a little farther here. The configuration you suggest works for lots of people, so this could be something peculiar to your hardware, which you haven't described at all. Please do note that the Forum Rules ask you to detail your hardware and configuration, which helps other community members spot possible trouble and produce better and more relevant answers to your questions.

 

[Red_November]

You really need to go a little farther here. The configuration you suggest works for lots of people, so this could be something peculiar to your hardware, which you haven't described at all. Please do note that the Forum Rules ask you to detail your hardware and configuration, which helps other community members spot possible trouble and produce better and more relevant answers to your questions.

Thank you for the hint please find full hardware details and happy to review any other hints

Code:

Chenbro nr12000 with the TYAN S5512 Motherboard
QUAD CORE E3-1230v2 3.3GHz CPU
16Gb of Ram
6 X SAS 6TB Drives
6 X SSD 1TB Drives
1 X SDD 32Gb Boot Drive
LSI SAS2008 SAS Controller
Intel C204 SATA Controller
Onboard Intel 82574L Network Controller
PCIe HP 560SFP+ 10Gb 2-port Ethernet Server Adapter X520-DA2

Code:

root@truenas[~]# dmesg | grep -i ethernet
mlxSen: Mellanox Ethernet driver 3.5.2 (September 2019)
em0: Ethernet address: 00:e0:81:ed:34:66
em1: Ethernet address: 00:e0:81:ed:34:67

root@truenas[~]# pciconf -lv I grep -B3 network

em0@pci0:3:0:0: class=0x020000 card=0x00000000 chip=0x10d38086 rev=0x00 hdr=0x00
vendor = Intel Corporation
device = '82574L Gigabit Network Connection
class  = network
em1@pci0:4:0:0: class=0x020000 card=0x00000000 chip=0x10d38886 rev=0x00 hdr=0x00
vendor = Intel Corporation
device = 82574L 82574L Gigabit Network Connection
class  = network
root@truenas[~]#

Installing the PCIe HP 560SFP

root@truenas[/]# pciconf -lv I grep -83 network
ix0@pci0:1:0:0: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = '82599ES 10-Gigabit SFI/SFP+ Network Connection'
class  = network
ix1@pci0:1:0:1: class=0x020000 card=0x00038086 chip=0x10fb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation'
device = '82599ES 10-Gigabit SFI/SFP+ Network Connection'
class = network

root@truenas[/]# dmesg | grep -i ethernet
mlx5en: Mellanox Ethernet driver 3.5.2 (September 2019)
ix0: Ethernet address: 90:e2: ba: 82:99:1c
ix1: Ethernet address: 90:e2: ba: 82:99:1d
root@truenas[/]#

 

[jgreco]

You're putting an HP OEM version of the X520 in there? I wonder if the HP firmware is stomping on something...

What's happening there with the Mellanox?

 

[Red_November]

Thank you for the review can the HP X520 be flashed ? or is that out of the scope of this forum ?

The Mellanox is the HP X520 I can assign a IP address to it but can't connect to it just yet as I am waiting for a SFP for my switch to ship. So I can only use copper for connections on the onboard Intel 82574L. That's working fine until I plug the HP X520 in.

 

[jgreco]

No, the Mellanox is not the X520-DA2. An X520-DA2 is an Intel card.

 

[Red_November]

That is interesting, I am not sure where that card is getting picked up from. I was reading about fake cards and firmware. Bad flash ? I have one other card but it came from the same source.

 

[Red_November]

Ok so I did I did a yottamark verify on the card and got the following back, showing its a valid card but its listed as a X520-SR2 card does that make any difference. Still does not explain why the Mellanox drivers are being loaded

Product Name:

INTEL(R) ETH CONVERGED NTWK ADPTR X520-SR2

 

[Samuel Tai]

Ok so I did I did a yottamark verify on the card and got the following back, showing its a valid card but its listed as a X520-SR2 card does that make any difference. Still does not explain why the Mellanox drivers are being loaded

Product Name:

INTEL(R) ETH CONVERGED NTWK ADPTR X520-SR2

The YottaMark mismatch means you got a fake.

 

[Red_November]

The YottaMark mismatch means you got a fake.

Thanks Samuel, I have pulled the card again and the sticker is showing that it is listed as a X520-SR2, card not the X520-DA2 that the vendor had said they where sending me. But looking up the SR2 Vs the DA2 it just describe whether the card ships with optics or DACs

The grep -B3 network command I think is giving me the line about the Mellanox driver due to the word network in the line.

Or is there something else in the data I posted that is hinting it as a fake card ? Sorry just a confused person learning Linux commands as I go

 

[jgreco]

Not quite, SR2 vs DA2 describes whether it ships with optics or just empty SFP+ ports. There is no option for DACs.

However, Intel is relatively ... butt-retentive ... about stuff like marking their parts, so if it says SR2, then it shipped as an optical card. This probably doesn't mean much, as the card's SFP modules can be pulled, leaving you with a card that is essentially an X520-DA2 but identified by the ROM as an SR2 card. That itself may not be particularly concerning.

More curious, however, is that you've mentioned that it is an HP card, and OEM cards typically do not get the Yottamark. Mmm. Curious. Maybe you should compare it to some eBay photos of other HP cards.

Either way, I get the feeling that your mainboard is seeing something about the card and deciding to disable the onboard ethernets. PCIe lane shortages on E3 systems is definitely a thing, and it would be worth a look through the manual to see if insertion of a card in a particular slot, or of a particular type, might result in disabling of the onboard ports. Otherwise I am kinda leaning towards something in the HP firmware possibly stomping on the PCI configuration for the onboards somehow.

 

[Red_November]

Thanks for the following up Jgreco, and for explaining in more detail about the part number.

So the whole confusion about the HP card is that the seller listed that it was a HP card and even had screens shots of the card , They where selling a number of cards together in the lot. When the card came I did not even look at the card until I started having issues. It was then only this weekend that I saw that it was not a rebranded HP or OEM HP Card it was in fact a straight branded Intel X520-SR2 card with Intel Optics.

I think you might be on to something about the mainboard I will go and read the manual and see if there is an option about it maybe disabling the onboard network card.

 

[Red_November]

You where right

There is a setting in the BIOS of my motherboard that will disable the on-board NIC when a network card is added.

To change this I went in to the BIOS "Advanced" -> "Onboard Device Configuration" and changed the "Dynamic Onboard LAN Control" to "Disabled"

Rebooted and saw the device listed, set the port to DHCP and TrueNas was able to assign the web interface an IP address and then I was able to login and manage TrueNas.