Interrupt storm takes down entire LAN

[prophoto]

I am running FreeNAS-11.3-U4.1 on an HP Z820 with 32GB of RAM and dual Xeon 2.4ghz 6 core processors. When I plug in a second NIC I get an interrupt storm and it shows errors on the terminal. It also takes down my ENTIRE LAN. Everything disconnects even my cameras. I've tried the two onboard NICs and the PCI card and one onboard NIC. Suggestions?

 

[prophoto]

Bump

 

[Redcoat]

Are the two NICs on different subnets?

 

[prophoto]

Are the two NICs on different subnets?

Thanks for your reply. Negative, they are on the same subnet. Ideally I wanted my VMs and jails to be on a separate NIC from the shares.

 

[Newfoundland.Republic]

Hmmm... Are you assigning an IP to the second NIC? The VMs and jails can supply their own IPs.

 

[SweetAndLow]

Chelsio T520-CR + Ubiquity SFP+ Switch + FreeNAS 11.2 + Link Aggregation

Ok, so let's start by saying I don't work with this stuff for a job and it's basically all brand new to me. After some time fighting things I have finally gotten everything set up and running(I think, it's only been a couple hours). I wanted to make a post documenting the struggles I had and the...

www.ixsystems.com

Only time I have made this happen was when I had 2 nic hooked up and I restarted a jail.

 

[prophoto]

At first I had them both on DHCP but since then I found the doc that said only one NIC can be DHCP so I tried setting the second as static. No change.

 

[Samuel Tai]

In general, with 2 NICs on the same subnet, you should use link aggregation to create a single virtual NIC or active/standby pair. Otherwise, having 2 NICs on the same subnet isn't a supported FreeNAS configuration, as for jails/VMs, one NIC will be bridged, and the other won't.

 

[Redcoat]

Thanks for your reply. Negative, they are on the same subnet. Ideally I wanted my VMs and jails to be on a separate NIC from the shares.

OK, as you have seen already from Samuel - above - where I was headed is that they need link aggregation to be on the same subnet.

 

[prophoto]

In general, with 2 NICs on the same subnet, you should use link aggregation to create a single virtual NIC or active/standby pair. Otherwise, having 2 NICs on the same subnet isn't a supported FreeNAS configuration, as for jails/VMs, one NIC will be bridged, and the other won't.

So you are saying this is the cause of the interrupt storm and the LAN going down? Or are you saying its not supported and may or may not work?

 

[prophoto]

This link states "make sure that all interfaces to use in the lagg are not manually configured", does that refer to the IP address?

8. Network — FreeNAS®11.2-U3 User Guide Table of Contents

www.ixsystems.com

 

[prophoto]

Also this documentation is wrong, there is no such page.

"To create a link aggregation, go to Network ‣ Link Aggregations and click ADD"

 

[Redcoat]

Try Network, Interfaces as described in the current version's docs at https://www.ixsystems.com/documentation/freenas/11.3-U4/network.html?highlight=link aggregation#interfaces (though I'm not sure that LAGG is what you need...).

 

[Samuel Tai]

This link states "make sure that all interfaces to use in the lagg are not manually configured", does that refer to the IP address?

8. Network — FreeNAS®11.2-U3 User Guide Table of Contents

www.ixsystems.com

Correct. This is because the lagg interface will have the IP, not the members of the LAGG.

 

[prophoto]

Tried adding an LACP LAGG. Failed miserably. Took me an hour to get FreeNAS back online. Is there a better way to configure?

 

[jgreco]

I am running FreeNAS-11.3-U4.1 on an HP Z820 with 32GB of RAM and dual Xeon 2.4ghz 6 core processors. When I plug in a second NIC I get an interrupt storm and it shows errors on the terminal. It also takes down my ENTIRE LAN. Everything disconnects even my cameras. I've tried the two onboard NICs and the PCI card and one onboard NIC. Suggestions?

You can get into bad networking situations when you put multiple interfaces on the same network. If your switch doesn't support or isn't configured for STP, you can easily get a forwarding loop that would cause an interrupt storm and take down your switch too.

https://www.ixsystems.com/community/threads/lacp-friend-or-foe.30541/

This isn't intended to derail the existing discussion but rather just to provide some background for what may be going on.

 

[prophoto]

Tried again today with no luck. Help!

Steps....
1. Created Aggregated port on my UniFi Switch 24 Version 4.3.20.11298
2. Created LAGG in GUI. Forced to run test. Once it fails settings get crapped out since it 'fails'. Goes back to old settings and LAGG is gone from GUI
3. Tried creating LAGG at terminal. Can't choose the proper 2 NICs (em0 and em1) only the one that I don't want shows up (bgem0)
4. Tried rebooting. and all steps again. Nothing.

 

[prophoto]

Forgot to mention I also tried this with FreeNAS plugged into configured LAGG ports on switch but it drops connection and can't get to GUI.

 

[prophoto]

Ok this time I deleted all config from all NICs then configured LAGG from terminal. Network was very unstable.

 

[prophoto]

Bump