Mellanox MNPA19-XTR not detected after reboot...

[JohnDonaldson]

Hello,


I recently upgraded my FreeNas server with one of these Mellanox MNPA19-XTR ConnectX-2 network cards. I run a direct fiber line from my server to my main desktop. This setup seemed to work perfectly at the start, even after giving the interface a IP and a subnetmask in the range of the other card in my main desktop, everything was still in working order (I even was able to access my shares at amazing speeds :D). There is also a second gigabit connection just to access the internet on server. The problems started occurring after I rebooted the FreeNas server. For some reason the mellanox interface wasn't being detected? I went to the FreeNas webinterface using the second interface (gigabit internet connection) and in the networking tab the mellanox card was detected but had a "Unknown Media Status". I did some research and found this old link describing a similar problem (with a older version of FreeNas - comment is a the bottom) Unix StackExchange. Is there anybody with more experience than me about this subject that could help out a hand? I find it pretty strange that the card just disappears from the system after a reboot, so I can conclude that FreeNas does support my specific cards. And that I maybe am missing some required drivers or specific settings to make this work. I also did a "pciconf -lv" in the FreeBSD shell and the mellanox card showed up, but I was confused by the information it gave for the card:
vendor = 'Mellanox Technologies'
device = 'MT25408 [ConnectX IB Flash Recovery]'
class = memory

There were no other mentions about any other mellanox products.


Picture of the interfaces FreeNas detected in the network interface.

My sincere apologies for my English (not a native speaker)
Thank you for any responses.

 

[Chris Moore]

I find it pretty strange that the card just disappears from the system after a reboot,

This does not correctly describe the situation. The card is not disappearing, it has an "Unknown Media Status", and that is a fault with the Mellanox card, not FreeNAS. I have had the same problem with a Mellanox card installed in a Windows system. Easy solution was to unplug and replug the SFP+ connector at the back of the card. Long term, I solved it by using Chelsio card instead.
https://www.ebay.com/itm/LOT-2-CC2-...-x8-LP-Server-Adapter-Card-w-SFP/132925940831

 

[JohnDonaldson]

Oke i'll try that. Maybe I can try to swap out my Mellanox card from my desktop to my server and see if the same problem occurs. Thank you I didn't knew this was related to the card. Weird that it works at the start and not after a reboot ? I might also check out those Chelsio cards.

Greetings.

 

[Chris Moore]

Weird that it works at the start and not after a reboot ?

The card I have in my Windows system only has a problem occasionally, not on every reboot. In fact, it hasn't given me a problem since the last time I removed and reinserted the SFP+ connector and I have rebooted almost ten times since then. I have a switch that has four SFP+ ports in addition to the 1Gb ports, so I have two FreeNAS systems and two other computers connected to the 10Gb network and I only have trouble with the ones using Mellanox cards and only occasionally, not every time.

 

[soft_reset]

Did you ever figure this one out, John? I'm having the same issues with two different ConnectX3 cards, single and dual port. They work great when they work, and in esxi I had no issues at all so far, but when booting bare metal it's a total crapshoot.
Especially on Windows 10 it feels like some weeks it works and some weeks it doesn't... though that might be because I reboot that machine a lot more.

 

[dougs]

I can confirm that by reseating the SPF+ connector solved the issue. I have rebooted a few times since then and each time FreeBSD recognizes the card! Go figure!