SolarFlare 6122F-SR 2nd port won't turn on once Truenas boots

difggy

Cadet
Joined
May 22, 2023
Messages
3
Hello everyone. New hear to the forums. My system is in my sig. Here's the issue I'm having.
I recently installed a SF6122F-SR as recommended in the 10g Primer article. The purpose of this was to create a faster connection (direct) from my workstation to the NAS. For the main LAN connection I use the onboard nic which I believe is from Killer Networks, static IP works fine. On the SF6122 (sfxge1) I set a static IP of 10.1.1.11 and on my workstation set 10.1.1.10. Used 10GTek transceivers on both ends. Connects just fine.

The problem with the second port (sfxge0). I have the second port connected to the SFP port on my Unifi 16 port PoE switch. I used 1.2G 850nm transceivers on both ends and an OM3 MM fiber cable. I've read that SFP+ is backwards compatible with SFP so they should connect at 1G. I checked to make sure that light is traveling through the cable, I can see the laser through each cable on both ends (when one end is connected to a device of course).

When I first turn the server on, both link lights are on on the NIC. But once Truenas completes the boot process, the 2nd port shuts off, and dmesg shows in order "sfxge0 link state down" while all the other network interfaces show the usual boot process "link state down, link state up" and they stay up. But sfxge0 stays down. I tried setting the interface to DHCP, doesn't come up. I tried setting a static IP, stays down. I even tried ifconfig sfxge0 up in the console, there's a slight delay, then the command completes and returns to the console, but no link light, and interface still says link state down. The interface is not being passed through in tunables.

As a sanity check, I tried swapping the ports. I plug the cable from the switch to the other port (sfxge1) that my workstation is plugged into, no lights. Plug my workstation into sfxge0. link comes up. I even tried swapping the cables. So I know that the ports on the SF6122 are good and the cables are good.


I'm fairly new to the fiber world so I'm not quite sure how to further troubleshoot, and I'm not sure this is a physical problem since when the server powers on the lights are on, but one turns off after the OS boots.
 

difggy

Cadet
Joined
May 22, 2023
Messages
3
Sorry since I'm new I can't edit and forgot to add an important detail:
I tried connecting my workstation directly to both SFP ports on the switch, and it worked fine, auto negotiates to 1G.
 
Top