SOLVED Networking issues with 2 NICs on motherboard

[Hazimil]

Hi All, as per my problem mentioned in this post, which was logged as bug. Basically in FreeNAS 11.3-U1 my server kept crashing when the Jails were loaded (via load config), finally, I managed to quickly stop the Jails before it crashed and disabled auto-start; this stopped my server from crashing.

The response I had from IX Systems was:

It is difficult to diagnose without seeing state of the problem. I haven't tried to load your config onto real system. But it looks wrong even with jails disables to see two network interfaces with IP addresses from the same segment. You should either disconnect one of them, or, if you really want to use both – configure LACP aggregation on both switch and FreeNAS. Depending how your jails are configured, I suspect you may create some network loop. Please clean up your network config and then les us know if the problem still exist and respective details.

Unfortunately networking is one area in which I am weak, however the sever has been running for 3-years now with the two network ports active on the separate IP numbers, it's only now with v11.3 that I seem to have a problem. My network ports are assigned the following IP numbers by my router:

192.168.0.100/24
192.168.0.101/24
192.168.0.102 (this is the BMC port)

Happy to accept its network, as this is what I was expecting it to be as it was linked to the Jails starting up and most probably allocating IP/mac address (which I understand the process has has changed in v11.3). However, I'm not sure what IX means by "two network interfaces with IP addresses from the same segment" nor what "LACP aggregation" is, can you point me to any resources to assist with this?

Finally, before I bring up my Jails, would it be useful for me to edit them and "update if necessary" the networking aspects of the Jail configuration?

Any help would be appreciated as at the moment I have no Jails running :(

EDIT: I see there are a couple of pinned threads about this in this forum, but it's all a bit confusing me - might have to read it a few times to figure it out, but any pointers welcomed.

Thanks,
Jonathan

 

[Basil Hendroff]

However, I'm not sure what IX means by "two network interfaces with IP addresses from the same segment" nor what "LACP aggregation" is, can you point me to any resources to assist with this?

This and preceding sections of the 11.3-U1 User Guide 9.2.2. Link Aggregations

 

[Hazimil]

OK, from my first read through, if I want to use both NICs, then I need to enable LACP aggregation on FreeNAS and also on my switch.

My current switch is a NETGEAR 16-Port Gigabit Ethernet Unmanaged Switch (JGS516) which means it doesn't support LACP aggregation.

So therefore, my options are to replace the switch with a managed option, and then configure LACP aggregation OR disable one of the NICs on the motherboard? To move forward with getting my Jails back up and running, I think the best option is to disabled one of the NICs whilst I look at managed switch options. Have I understood this correctly?

EDIT: Any recommendations for a rackmounted 16-port Gigabit managed switch (for home user)?

With regard to disabling a NIC, is this done in FreeNAS or should I see if it can be disabled within the system BIOS? Bit of a steep learning curve this one for me!

I'm assuming that the fact that I've not had an issue over the last 3 years with the server with both NICs live is just down to being lucky!

Thanks,
Jonathan

 

[Basil Hendroff]

Sorry, I've got no experience with LACP, only what I've read. @Adrian seems to have a good handle on this and appears to be across current issues as well. See Erratic 11.3-RELEASE lagg/failover behaviour

 

[Kcaj]

With regard to disabling a NIC, is this done in FreeNAS or should I see if it can be disabled within the system BIOS? Bit of a steep learning curve this one for me!

Just unplug one of the cables ;)

Have a read of this if you haven't already.

 

[Adrian]

LACP aggregation probably wouldn't do you much good.
You generally need a large number of clients for it to work well.
With just a few clients you can find that most of them are using the same NIC.

Worse, FreeBSD has for some time needed a switch that does Active LACP, and some switches (e.g. my Netgear* GS724Tv4) only does Passive.
You could configure the lagg for failover, but do ensure that both NICs have the same IP address,
either by using static addresses or arranging this in your DHCP server if it supports this. pfSense does.

I have not tried the other lagg options (loadbalance, roundrobin and broadcast). You could experiment. Or as @Kcaj suggests just unplug one NIC.
Do read @jgreco at this, you will be much better informed. Also man lagg or see an online man page.

* My experience of Netgear devices is that the hardware is fine, but the software can be awful. I no longer buy anything from Netgear that is
configurable. Aside from the managed switch, I have had severe trouble with a Nighthawk X6 Smart Wifi Router (R8000) and
a Wi-Fi Range Extender EX2700. I have quite a few of their unmanaged switches.

 

[Hazimil]

Hi All, thanks for the replies. I have read the useful posts by @jgreco and understand a little bit more about it, only a little, but enough to understand the basics. I have opted for the simple option and just unplugged one of the cables for now - lets see if that fixes my Jail and server crashing issue.

Once the server is stable, I will go back and do more research on this, as mainly I was thinking of using the port(s) as fallback in case one fails - bandwidth increase is lesser of an issue due to lack of connected clients.

I've worked in IT for 30 years and have tried to "ignore networking" and leave it to the geeks, I might have to get my hands dirty now! :)

Jonathan

 

[Basil Hendroff]

Another resource by @jgreco that may be relevant to this discussion LACP ... friend or foe?

 

[Hazimil]

Seems to be working OK now I've unplugged one of the network cables. Wonder why it worked with no errors/crashes for 3 years and only now started with the 11.3 update?

Oh well, resolved for now and Jails back and working :)

Thanks all for the help.

Jonathan.