FreeNAS Box appears to be causing packet storm ?

[NightWolfx03]

I have a FreeNAS box I recently added to my network, everything has been working fine for the most part. Today I noticed something strange, all the lights for activity on my HP Procurve switch where either flashing alot or on solid ( which means there's alot of traffic) as I was trying to figure out what was going on, I loaded Wire Shark on one of my machines and noticed that I was seeing a lot of traffic naming files I was copying. So I went and paused the file copy and all the lights on the switch ports settled down and resumed a normal amount of blinking, I restarted the copy and for a while it acted fine then I started seeing all the activity again. It seemed to be generating something like 10 ~ 30 mbps of traffic to every machine on my network. So I messed around with it some more and I disabled the secondary network interface on the FreeNAS machine and so far, it seems to have stopped. Does anyone have any idea's why having a second interface enabled would be doing this ? I didn't configure any of the special protocols, I just had 2 interfaces enable each with a separate IP address. When this was happening I was coping files from a Windows 7 machine to the FreeNAS box.

The FreeNAS install is less than 2 months old, and the hardware is a slightly dated dual 1366 Socket Supermicro motherboard. I can provide more information if needed.

 

[Patrick M. Hausen]

Any chance you created a bridge interface with both physical interfaces as members?

 

[NightWolfx03]

Not that I know of, they were just setup as individual interfaces with their own IP addresses. A friend of mine was saying something about the FreeNAS / BSD network stack might not like it, so I am now testing it as a Load Balanced Aggregation now instead, which I probably should have done from the start.

 

[Patrick M. Hausen]

Did you use two interfaces with addresses from the same subnet? That's not how IP works. Nothing BSD about it.
One broadcast domain, one (or more) layer 3 prefix(es). And if you used different networks, you should connect them to different switches or partition the switch with e.g. port based VLANs.

HTH,
Patrick

 

[NightWolfx03]

Yeah It was on the same network. I think I just did a silly and setup IPs for both the interfaces instead of setting them up in link aggregation.

I removed the settings from both interfaces, then created a lagg using both and configured the lagg with a single IP.

I initially set it up in 'loadbalance' mode ( about an hour ago ) and it was working for a bit and then it started being weird again. I'm going to check the switch config when I get a moment, but for now I have switched it to 'failover' which seems to be working just fine.

 

[mountaintime]

Any chance you created a bridge interface with both physical interfaces as members?

Hi Patrick,

I ran into the multiple interfaces on a bridge and believe I know what is happening, but I am hoping for some help on how to control the configuration.

• FreeNAS 11.3-U3.1
• ix0 - 10gbe Main connection to freenas
• igb0-igb5 1gbe Intel NIC interfaces exist, all unconfigured
• a VM is attached to igb(X) on a different subnet to run IPcamera software VM which uses a bit of bandwidth and I'd like to not have that overhead on my main file service interface.
• Plex Plugin running through VNET default settings.

On reboot, the vnet Bridge0 will add igb(X) to the members causing my network to crash. I physically disconnect the nic I can then go reconfigure the VM to a different igb(X+1) and everything is fine. Stopping the Jail and restarting the jail does not change the Bridge0 configuration, but rebooting removes igb(X) then adds the new igb(X+1) to its members which crashes the network.

• Is there a way to configure Bridge0 so that it is locked to only ix0 and sustains reboots?

example ifconfig output is below: in this case igb0 was set to a functioning VM and the system rebooted; Bridge0 added it as a member, and network crashed. igb0 was disconnected, VM nic changed to igb4 and VM running. Help would be greatly appreciated!

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:96:
nd6 options=1<PERFORMNUD>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 12 priority 128 path cost 2000
member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 7 priority 128 path cost 2000
member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 2000000
vnet0.1: flags=8942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: plex as nic: epair0b
options=8<VLAN_MTU>
ether d0:50:99:
hwaddr 02:db:
nd6 options=1<PERFORMNUD>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
groups: epair
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: Attached to ubuntuVM
options=80000<LINKSTATE>
ether 00:bd:
hwaddr 00:bd:
nd6 options=1<PERFORMNUD>
media: Ethernet autoselect
status: active
groups: tap
Opened by PID 3886
bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:96:
nd6 options=1<PERFORMNUD>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: igb4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 20000
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 9 priority 128 path cost 2000000

 

[Patrick M. Hausen]

Of course ;)

Disable automatic startup of all VMs and jails
Reboot
Go to Network --> Interfaces
Click Add
Explicitly create bridge0 with the desired member interface
Same for bridge1
Associate VMs and jails to bridge0 or bridge1 as needed

BTW: the FreeBSD handbook recommends putting the IP address on the bridge interface and configuring the members just "up".
Also you should disable hardware acceleration features (there's a checkbox for that) for the physical interfaces.

HTH,
Patrick

 

[mountaintime]

Of course ;)

Disable automatic startup of all VMs and jails
Reboot
Go to Network --> Interfaces
Click Add
Explicitly create bridge0 with the desired member interface
Same for bridge1
Associate VMs and jails to bridge0 or bridge1 as needed

BTW: the FreeBSD handbook recommends putting the IP address on the bridge interface and configuring the members just "up".
Also you should disable hardware acceleration features (there's a checkbox for that) for the physical interfaces.

HTH,
Patrick

Hi Patrick,

Thank you for the quick reply, and thank you so much I had been searching this issue for weeks. I had tried creating bridges in the past, but was not rebooting in this sequence, and the jail wouldn't get any IP.

I stopped jails/VMs, rebooted, created the interfaces bridge0,1,4,5555 (testing) all associated with igb0,1,4,5. edited the jails to vnet0:bridge0,1, and VM to bridge4 (previously set to igb4). After saving the config I rebooted and checked ifconfig via ssh. all looks as expected. interfaces and bridges were left unconfigured, interfaces had hardware offloading disabled. After the first reboot, bridge0 still added ix0 in and crashed everything.

I double checked all the settings and set the jail default vnet interface to "none" and 2-3 reboots and testing different configured bridges later, the double interface issue seems to be resolved. I can see each service is communicating via the appropriate nic.

I am really glad to get this resolved, I guess I never rebooted multiple times before and was always getting ix0 adding into the first service that got started. Learning everyday....

Thank you again!

Kevin