10g networking card causes FreeNas "Fatal Trap 12"

[espinobj]

FreeNAS-11.2-U3
MoBo: Asus MSA97 LE R2.0
CPU: AMD FX(tm)-4170 Quad-Core Processor (4 cores)
Ram: 16GB
Boot: Dual Samsung 3.1 external USB
HDD x6: Seagate Barracuda ST2000DM008

This has been my file & Plex server for the better part of the month. No issues! Lot of tinkering rearranging etc etc. All with no issues at all. FreeNAS is a beast!


Until...
I got in my 10gb network upgrade. (Help me)
2x - Emulex OCE 10102 dual SFP+ 10gb ports
2x - A7EL-SN85-ADMA 850nm transceiver
2x - FTLX8574D3BCL 850nm transceiver
(two different transceivers because that was what I had on hand. Only intent to use one pair for now until my skill level increases and I bump my whole home to 10g)

**Note on the same 10g card I also see a sticker for HP CN1000E through my research it appears to be no difference just a buy out of Emulex I assume
Placing this card into me FreeNAS server causes a Fatal trap 12: page fault while in kernel mode removing the card and system boots normally.

Is this simply an incompatibility between FreeNAS and my network card? Does anyone have experience with these cards or similar? Im extremely green when it comes to this side of enterprise networking and I could use the extra help and knowledge. Id like a direct 10gb link to my FreeNas box via one of my Fiber 10g SFP's. My profession is in structured cabling in data centers and large scale construction projects with expertise in fiber optic cabling.

 

[jgreco]

We have a nice 10 Gig Networking Primer and discussion thread that you should check out.

https://www.ixsystems.com/community/threads/10-gig-networking-primer.25749/

It talks about all the things that work and all the things that don't. There's at least as much information in the discussion thread as there is in the Primer itself. Make sure to click on "Show: Current Version" or to look at the resource, not just the discussion thread.

If you're familiar with the fiber side of all this, I will caution you not to breeze over this too quickly because there's a lot of little information nuggets buried in between home user confusion and guidance on fiber.

 

[espinobj]

I swear I read that primer twice and didnt see this little comment.

@Borja Marcos notes: Beware the Emulex "oce" cards - serious issues with them, panics when moving some traffic. There is a patch (see relevant discussions on the freebsd-net mailing list) but the stock driver crashes badly.

Truth be told Im having massive issues simply getting the windows side of the card to work. $$ down the drain. Looking into Chelsio line of cards.

 

[jgreco]

I swear I read that primer twice and didnt see this little comment.

Sorry! :-( Seriously, I do all this stuff just to save everyone else the annoyance of learning the hard way. I hate to fail.

Truth be told Im having massive issues simply getting the windows side of the card to work. $$ down the drain. Looking into Chelsio line of cards.

Apparently the Chelsio's have dropped quite a bit on eBay. The other good deal I've been seeing are the SolarFlares (SFN6122). I just picked up several at less than $30/ea. Now, warning, seriously, I HAVE NOT TRIED these on FreeNAS, but after kicking the tires on them for a long time on a test ESXi box, they're cheap, they've got better lights than the Intels, they eat half the power of the Intels, and I can't find a single damn bad thing to say about them, which for me is pretty remarkable. I won't tell you how to spend your money, but if you run across one cheap, I'd trust it more than the Emulex.

 

[Qumefox]

The emulex OCE cards (at least the pair of HP NC550SFP's I have) worked flawlessly in freenas 10 after they were flashed to the latest firmware. They ran trouble free for well over a year before I retired that machine. the OCE driver issues apparently started with freenas 11.1 and made them extremely flaky. In 11.2 they refuse to boot at all. They'll give fatal trap 12's on boot if installed in the machines, no exceptions from what I can tell. (tried on four different motherboards so far)

From what little I know. it seems like the oce driver is 3rd party and provided by broadcom I think? (not sure who owns emulex now) which means its not something that can be fixed in either freeBSD or freenas, and as old as these cards are, i'll be amazed if we ever see an actual working driver again.

 

[jgreco]

The problem with vendor-supplied drivers is that at some point, it is usually the case that the vendor stops selling the card, at which point the incentive to update drivers is greatly reduced. The engineers responsible for the driver may move on to other projects, or even on to new companies, leaving the vendor without an easy way to update the driver, even if they wanted to. Then the company gets acquired or merged or goes out of business and there you are, EOL.

The problem with drivers developed by third parties is that they frequently do not have the depth of knowledge allowed by chipset documentation; many drivers are reverse-engineered without formal documentation. This is fine as long as some hardware hacker is interested in pursuing the challenge, but this does mean that they need to have at least one system and one device to test with. That isn't really sufficient for thorough driver development, as there are various issues on different hardware platforms that affect drivers, so these have historically been somewhat hit-or-miss.

Sorry if that all sounds a bit pessimistic.

 

[espinobj]

Exactly what happen with me those cards couldn't even get the window side to work. So ultimately I went to a set of Chelsio n320e Got them in two days ago and boom. Fail again!

This time it seems that FreeNAS is good and it was plug and play besides my IP configs. The problem was Windows 10- BSOD. After diving into it plagued by outdated drivers again. The only Chelsio drivers that I found that are supported in a windows 10 environment is the T5 and T6 series not my super cheap ebay T3 :( and I'm not able to shell out $350 for one at the moment.

So currently I'm in the process of seeing If i can use a VM of linux to achieve a 10gb link. And have read in other forums that the mellanox connectx-4 is a decent Windows choice but I see a recurring theme here, divers regardless of you main OS, will make or break you.

 

[Qumefox]

I finally broke down and got a pair of intel x520-da2's to replace my emulex cards. Luckily I found a few for $50 each, because honestly after 8 new drives and a new HBA, i'm tired of throwing money and time at this thing. I have other projects I need to work on other than fighting with a storage server.

 

[SweetAndLow]

Exactly what happen with me those cards couldn't even get the window side to work. So ultimately I went to a set of Chelsio n320e Got them in two days ago and boom. Fail again!

This time it seems that FreeNAS is good and it was plug and play besides my IP configs. The problem was Windows 10- BSOD. After diving into it plagued by outdated drivers again. The only Chelsio drivers that I found that are supported in a windows 10 environment is the T5 and T6 series not my super cheap ebay T3 :( and I'm not able to shell out $350 for one at the moment.

So currently I'm in the process of seeing If i can use a VM of linux to achieve a 10gb link. And have read in other forums that the mellanox connectx-4 is a decent Windows choice but I see a recurring theme here, divers regardless of you main OS, will make or break you.

My Chelsie t5 was $115 off eBay. That's pretty cheap if you all me.