SOLVED 10GbE Ethernet Not Found

[dddza]

Hello,

I have two FreeNAS machines running very similar Supermicro hardware.

Box 1:
FreeNAS-9.2.1.6-RELEASE-x64 (ddd1e39)
System Information
Manufacturer: Supermicro
Product Name: X9SCL/X9SCM
10GbE NIC: Supermicro AOC-STGN-I1S <- single port (Intel® 82599EN 10 Gigabit Ethernet Controller)

Box 2:
FreeNAS-9.2.1.6-RELEASE-x64 (ddd1e39)
System Information
Manufacturer: Supermicro
Product Name: X9DRi-LN4+/X9DR3-LN4+
10GbE NIC: Supermicro AOC-STGN-I2S <- dual port (Intel® 82599ES 10 Gigabit Ethernet Controller)

From initial install, Box 1 saw the 10GbE NIC and loaded it right away:

ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
options=407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO>
ether 00:25:90:e1:c2:e7
inet 10.12.13.153 netmask 0xffffff00 broadcast 10.12.13.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
status: active

However, Box 2 does not even find the card. I have tried moving it to different slots. Box 1 has 2 onboard NICs and Box 2 has 4 onboard. It finds all 4 onboard but just not the ix0/1 NIC.

I do not find anything about "ix0" or anything else related to the 10GbE NIC in dmesg.

I do not have a clue why this would be happening or what steps to take.

Thank you in advanced.

 

[Ericloewe]

Switch the cards between systems. If the problem follows the card, it's defective - replace it.

 

[joeschmuck]

Switch the cards between systems. If the problem follows the card, it's defective - replace it.

Great minds think alike.

If the problem remains with the second system (does not follow the card), then you need to do more troubleshooting. It could be a lot of things.

 

[dddza]

Thank you for the replies.

I have two brand new cards (other than the one in Box 1, as it's a live system) and both do not work... so we can rule out the card.

I can confirm the card is powered as the switch and light at the back of the card are both lit.

 

[joeschmuck]

Are those cards the same model with the dual ports? I'm curious if it's the card model or not. The other thing, and I'm grasping at straws here... You could disable some of the internal NIC ports and try again... maybe it's a IRQ issue?

 

[Ericloewe]

Any chance you can try a different OS on the same hardware?

 

[c32767a]

Are those cards the same model with the dual ports? I'm curious if it's the card model or not. The other thing, and I'm grasping at straws here... You could disable some of the internal NIC ports and try again... maybe it's a IRQ issue?

The single port card chipset has the EN suffix, the dual port card has the ES suffix. According to the FreeBSD networking lists, both chips are supported in 9.x.


As has been discussed elsewhere, there be dragons hiding under the covers of the ixgbe driver and TSO in FreeBSD 9.
It's probably too late now, but IMHO you would have been better served to get a SolarFlare or Chelsio card.

 

[dddza]

The single port card chipset has the EN suffix, the dual port card has the ES suffix. According to the FreeBSD networking lists, both chips are supported in 9.x.


As has been discussed elsewhere, there be dragons hiding under the covers of the ixgbe driver and TSO in FreeBSD 9.
It's probably too late now, but IMHO you would have been better served to get a SolarFlare or Chelsio card.

Yep, it is certainly too late as we have a bunch of these cards and they are working perfectly for our network - including the existing FreeNAS box..

Thanks for spotting that suffix, I have corrected it on my original post.

 

[dddza]

Any chance you can try a different OS on the same hardware?

I could try as not much else has been done yet.. I'll get CentOS installed and see what it finds.

 

[dddza]

Are those cards the same model with the dual ports? I'm curious if it's the card model or not. The other thing, and I'm grasping at straws here... You could disable some of the internal NIC ports and try again... maybe it's a IRQ issue?

Yep! Exact same cards just single port and the other is dual port. The card working in Box 1 is a Single port and the Dual in Box 2 is the one that is not found.

 

[dddza]

Are those cards the same model with the dual ports? I'm curious if it's the card model or not. The other thing, and I'm grasping at straws here... You could disable some of the internal NIC ports and try again... maybe it's a IRQ issue?

I think you were spot on with the IRQ issue.. however, although I tried the original slot and another, I didn't try the remaining 3. On my first attempt of swapping to another slot, it was found right away!

 

[Ericloewe]

I think you were spot on with the IRQ issue.. however, although I tried the original slot and another, I didn't try the remaining 3. On my first attempt of swapping to another slot, it was found right away!

Messing around with some PCI-e setting in BIOS might solve the problem on the original slot then.

 

[joeschmuck]

This brings me back to the days where we manually assigned IRQ's for add-on cards.

 

[dddza]

This brings me back to the days where we manually assigned IRQ's for add-on cards.

haha that’s for sure! I would have liked to play around more in the BIOS and find out what could have been done but unfortunately it's a replacement setup for a failed unit so I didn't have much time :(