TrueNAS wont boot with intel X710 NIC

[kenster]

so
I'm trying to upgrade my TrueNAS to 10GB SFP to take advantage of our newly installed networking.

The server is a Dell R740xd running TrueNAS-13.0-U3.1 and the NIC is Intel(R) Ethernet Converged Network Adapter X710

When I install the new NIC, TrueNAS booting hangs just before the drives mount. If there is resources I haven't found on this issue already please point me in the right direction, otherwise, I'm looking for suggestions.

The card does show correctly in LifeCycle controller Inventory

 

[sretalla]

Have you looked at the firmware on the card? perhaps it's a conflict with the driver version in CORE and the current firmware.

 

[Ericloewe]

rNDC or standard PCIe card?

 

[Elliot Dierksen]

I have a Cisco OEM'd XL710 in my server and it boots fine. Can you disable the option ROM on the XL710 in the BIOS? That might help.

 

[kenster]

I have a Cisco OEM'd XL710 in my server and it boots fine. Can you disable the option ROM on the XL710 in the BIOS? That might help.

I will give this a try, thanks!

rNDC or standard PCIe card?

it's PCIe

Have you looked at the firmware on the card? perhaps it's a conflict with the driver version in CORE and the current firmware.

How is this done? Do I pop it in, then upgrade the firmware via iDrac firmware updates?

 

[Ericloewe]

Since you're using a standard card, iDRAC would only help if it's a Dell-branded card.

Otherwise, you need to navigate the depths of Intel's website, find the update package for the X710, and figure out the steps of the byzantine updater tool. From the experience with my I350s, I remember you need to prepare the package on a FAT32 mass storage device, boot into the UEFI shell, and locate both the utility and the firmware blob.

With a Dell card, open iDRAC, Maintenance, Update, Manual Update, check "default server" and click the check for updates button. Then install all your firmware and semi-useful-to-useless utilities by checking their checkboxes.

 

[kenster]

Since you're using a standard card, iDRAC would only help if it's a Dell-branded card.

Otherwise, you need to navigate the depths of Intel's website, find the update package for the X710, and figure out the steps of the byzantine updater tool. From the experience with my I350s, I remember you need to prepare the package on a FAT32 mass storage device, boot into the UEFI shell, and locate both the utility and the firmware blob.

With a Dell card, open iDRAC, Maintenance, Update, Manual Update, check "default server" and click the check for updates button. Then install all your firmware and semi-useful-to-useless utilities by checking their checkboxes.

does the card need to be installed when firmware updates are done? it is a Dell card.

 

[Ericloewe]

Well, yes, how else would you be able to load the firmware onto the card? (Silly question, you could flash the EEPROM directly, but that's a pain at best and not at all the point)

 

[kenster]

Well, yes, how else would you be able to load the firmware onto the card? (Silly question, you could flash the EEPROM directly, but that's a pain at best and not at all the point)

yeah, and my TrueNAS won't boot when the card is installed . . .

 

[Ericloewe]

Well, you don't need TrueNAS to boot. Just do the update process and select the reboot immediately option.

 

[kenster]

got it. how did I not think of that.
thanks so much. I'll try this on Saturday outside of work hours.

 

[firesyde424]

We currently have a PowerEdge R740xd running TrueNAS Core 12.0-U8.1. It uses an Intel X710-DA2 and Intel XXV710-DA2 NIC. We've not see any specific issues booting. I'd be very curious if your Lifecycle controller has detected any issues.

As far as updating these cards, it can be updated directly from the server's iDRAC. Log into the iDRAC. Maintenance > System Update > Manual Update.

Here's a link for the latest firmware for the intel X710: https://dl.dell.com/FOLDER10646142M/1/Network_Firmware_1R0W0_WN64_22.5.7_A00.EXE

 

[kenster]

well, no joy.
I updated all the firmware in iDrac, the NIC, the backplane, BIOS and I still can't get the system to boot with the Intel card in it.

In iDrac the card shows connectivity when it's plugged into the network, so it doesn't seem like a hardware issue. I think it's inside of Freenas that the issue lies.

 

[Ericloewe]

It was a bit of a long-shot, but it should be working... Have you tried other OSes?

 

[kenster]

It was a bit of a long-shot, but it should be working... Have you tried other OSes?

Not this specific card. I have a matching NIC in my XCP NG server working fine.

Is the simplest solution a different NIC? If so, which one?

 

[Ericloewe]

Thing is, it should be working, so I'm hesitant to suggest going out and buying a different card. If you happen to have something else on hand, it's definitely worth a shot. It would also be interesting to switch the two cards around to definitively rule out a hardware defect.

 

[kenster]

Thing is, it should be working, so I'm hesitant to suggest going out and buying a different card. If you happen to have something else on hand, it's definitely worth a shot. It would also be interesting to switch the two cards around to definitively rule out a hardware defect.

Agreed. I will try that.

Is there any way to troubleshoot the boot process? I did a quick peek at /var/log/messages but didn’t see a lot

 

[firesyde424]

Not this specific card. I have a matching NIC in my XCP NG server working fine.

Is the simplest solution a different NIC? If so, which one?

At this point, I'm thinking you might have an issue with the card? I don't know. I use Intel cards in something like 15 TrueNAS Core servers. They just work and have been reliable for a very long time.

 

[Arwen]

If I remember correctly, the Intel x710 type Ethernet cards have multi-functions. They can, depending on card, firmware configuration and media ports:

• Act as Fibre Channel HBAs, (was that over Ethernet media? I don't remember...)
• Divide up a single physical port into multiple virtual Ethernet ports, assignable to different VMs
• Allow single physical connector to break out to multiple lower speed connectors, (like 40Gbps to 4 x 10Gbps ports)

So, my suggestion is to see if their is a card configuration option and configure it for plain Ethernet. Intel might have such a utility available, or perhaps it's in some kind of boot time BIOS for the card.

 

[Ericloewe]

Act as Fibre Channel HBAs, (was that over Ethernet media? I don't remember...)

I think the X710 dropped that, and that the X520 did support it.

Divide up a single physical port into multiple virtual Ethernet ports, assignable to different VMs

SR-IOV, very useful. Also a feature on the 82599/X520, but the X710 supports more Virtual Functions. If SR-IOV support is enabled in the card's firmware (should be configurable from the system firmware setup menu), it's worth a shot trying to disable it if you won't be using it. Not super likely, though. It's available on all cards unless disabled in vendor firmware.

Allow single physical connector to break out to multiple lower speed connectors, (like 40Gbps to 4 x 10Gbps ports)

That's the XL710, but it's still plain Ethernet.