ports are not ACTIVE on LAGG

[Harrison]

Hello,
Since updating to 11.1-U6 on our iXSystems (Supermicro) FreeNAS-Certified-4U with 4x NIC's +IPMI, there are continual critical warnings like this "CRITICAL: Aug. 25, 2018, 8:28 a.m. - These ports are not ACTIVE on LAGG interface lagg0: igb1, igb2, igb3. Please check cabling and switch."

I've tried removing the (LACP) lagg and rebuilding both via the GUI and terminal but the same warnings still appear.
Currently the GUI setup is like this:

The corresponding switch is a CISCO SG300 managed switch, which automatically detects the lagg. Current settings:

I've been running this exact same setup for the past same 4 years with out issue - till now.
Here's hoping someone has a little magic to share.
Thanks, Nick.

 

[Harrison]

Realised this issue may have been developing for a while as I've previously seen the NIC's going up/down like this:
Aug 25 08:51:50 WEINIGFS kernel: igb3: link state changed to DOWN
Aug 25 08:51:50 WEINIGFS kernel: igb3: link state changed to DOWN
Aug 25 08:51:50 WEINIGFS kernel: igb2: link state changed to DOWN
Aug 25 08:51:50 WEINIGFS kernel: igb2: link state changed to DOWN
Aug 25 08:51:54 WEINIGFS kernel: igb3: link state changed to UP
Aug 25 08:51:54 WEINIGFS kernel: igb3: link state changed to UP
Aug 25 08:51:54 WEINIGFS kernel: igb2: link state changed to UP
Aug 25 08:51:54 WEINIGFS kernel: igb2: link state changed to UP

Our small office network has about 20-30 people accessing the file server. I know someone may suggest deleting the lagg and running with just one NIC. I'm not an IT tech but that realy bugs me as regressive.

 

[dlavigne]

Does the error only occur on U6 or does it also occur if you reboot back into U5?

 

[Harrison]

Hi dlavigne, that may be what I have to do - I'll have to wait for a quiet time to try this. Never the less I'll report back on the results. Thanks for the suggestion.

 

[Harrison]

Well I've tried rebooting into U5 and there was no change. So I'm back at U6 and I've gone to just one NIC (yes, I'll eat my own words for an easy life!) so the lagg error has gone.
I'll have this fight another day.

 

[short-stack]

Realised this issue may have been developing for a while as I've previously seen the NIC's going up/down like this:
Aug 25 08:51:50 WEINIGFS kernel: igb3: link state changed to DOWN
Aug 25 08:51:50 WEINIGFS kernel: igb3: link state changed to DOWN
Aug 25 08:51:50 WEINIGFS kernel: igb2: link state changed to DOWN
Aug 25 08:51:50 WEINIGFS kernel: igb2: link state changed to DOWN
Aug 25 08:51:54 WEINIGFS kernel: igb3: link state changed to UP
Aug 25 08:51:54 WEINIGFS kernel: igb3: link state changed to UP
Aug 25 08:51:54 WEINIGFS kernel: igb2: link state changed to UP
Aug 25 08:51:54 WEINIGFS kernel: igb2: link state changed to UP

Our small office network has about 20-30 people accessing the file server. I know someone may suggest deleting the lagg and running with just one NIC. I'm not an IT tech but that realy bugs me as regressive.

Is it only igb2 and 3 in the logs going up and down? They are your inactive members of the LACP group, so they might be going up and down as they are being talked to.

Can you try to set up the LAGG with only 2 of the interfaces? I've not seen any of these issues, but I only have 2 members. It could be something strange with how the Cisco switch marks two of the ports active.

 

[kdragon75]

Ill bet if you do a LAG with two ports, one goes up and down. I'm guessing the switch is using half the group for failover. Try your 4 port LAG and unplug igb0 and see if 2 or 3 come up and stay up. The CISCO SG300 is a SOHO switch and I find they univeralsy suck.

 

[purduephotog]

Oddly enough I'm having nearly an identical problem. First tech support said it was my copper DAC, so I just dropped a grand on fibre. Now fibre only shows the same 1 port active and is bouncing up and down.

If you ever decide to revisit please tell me if you got it working

Edit: nearly, not breakfast. I'm really hating this new phone.

 

[Constantin]

I also had lagg issues that drove me to exasperation. I eventually gave up on having a lagg failover work.

 

[purduephotog]

I also had lagg issues that drove me to exasperation. I eventually gave up on having a lagg failover work.

Thank you. I bloody hope that's not my outcome. But hey...

 

[Elliot Dierksen]

As a CLI biggot, I hate the GUI only switches, but I do have a couple thoughts. Are you forcing speed and duplex on either side? Are you using Cat6 or better cables? Can you disable auto-mdix (auto cross over) on either side? Seeing the interface physically flap like that can happen when each side is trying to do auto-crossover. If you have the interfaces up but line protocol is down, that means that LACP negotiation has failed. At least one side needs to be doing active LACP negotiation, and I am pretty sure FreeNAS does that. I am not entirely that sure what it means, but I would also try checking 'reactive suspended LAG' box. I have LAGG's working in FreeNAS to HP, Cisco, and H3C (re-badged as HP) switches.

 

[purduephotog]

As a CLI biggot, I hate the GUI only switches, but I do have a couple thoughts. Are you forcing speed and duplex on either side? Are you using Cat6 or better cables? Can you disable auto-mdix (auto cross over) on either side? Seeing the interface physically flap like that can happen when each side is trying to do auto-crossover. If you have the interfaces up but line protocol is down, that means that LACP negotiation has failed. At least one side needs to be doing active LACP negotiation, and I am pretty sure FreeNAS does that. I am not entirely that sure what it means, but I would also try checking 'reactive suspended LAG' box. I have LAGG's working in FreeNAS to HP, Cisco, and H3C (re-badged as HP) switches.

Hey now, don't call yourself that. A GUI is an interface to commands. They're supposed to help us not forget, and make it easier to execute complex commands in order... But they're only as smart as the coder and only as reactive as the most asinine situation over can imagine...

That said, no, I'm not forcing any speeds. The cables are copper, DAC, and fibre. None have worked. I'm suspicious the truenas box has a bad driver.

Only the single line of the quad works.

I'll keep digging...