SOLVED link aggregation, can only select 1 interface

[silbro]

Hello everyone

First of all I wanted to say, that I am very satisfied with freenas :) very nice!

I wanted to create a link aggregation today. when i log into free nas via ssh and do ifconfig -a I get the following output:

Code:

em0: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=4019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
  ether 00:25:90:46:57:ff
  nd6 options=9<PERFORMNUD,IFDISABLED>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
  options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
  ether 00:25:90:46:57:fe
  inet 192.168.10.5 netmask 0xffffff00 broadcast 192.168.10.255
  nd6 options=9<PERFORMNUD,IFDISABLED>
  media: Ethernet autoselect (1000baseT <full-duplex>)
  status: active
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
  nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
  options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
  inet 127.0.0.1 netmask 0xff000000
  nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

igb0 is configured:

I have 3 LAN ports total. 1 IPMI and 2 network cards. The board I am using is a supermicro X10SLL-F board.
http://www.supermicro.com/products/motherboard/Xeon/C220/X10SLL-F.cfm

In freenas I can only see 1 network card and i can add another one which is em0 (but I think its the IPMI, I havent plugged it in and it sais it isnt, so that makes sense). even if I add the em0 one I cannot add the igb0 to the link aggregation:

How can I add this second Network card? (I'm guessing its a driver problem because the network cards arent the exact same build, could this be true?)

I would like to create a failover link aggregation with the used igb0 and the not yet added network card. (I have both plugged in by the way).

Thanks for any tips :)

 

[dlavigne]

If it's not showing up in ifconfig, it means the kernel isn't finding it (or its driver) during bootup. Any idea what the model and chipset is for the interface that isn't being found? Have you verified that the link light works (e.g. the NIC itself seems operational)?

 

[cyberjock]

If I'm not mistaken (and I could be) the following must be true for you to do what you want:

1. Both NICs must support lagg.
2. Both NICs must be unconfigured in the WebGUI. (em0 is listed by itself because igb0 is already configured).
3. Your network switch must properly support lagg

I've never seen an IPMI device that is an Intel card (which em0 is Intel). IPMI is NOT made available to the OS as part of its network ports. That is, you cannot send system data out of the IPMI port. So any concern you have with em0 being your IPMI card is unfounded.

Assuming everything "supports" lagg, I bet if you delete your igb0 configuration (which will default to dhcp at that point) will let you create the lagg without problems. ;)

 

[Ericloewe]

If I'm not mistaken (and I could be) the following must be true for you to do what you want:

1. Both NICs must support lagg.
2. Both NICs must be unconfigured in the WebGUI. (em0 is listed by itself because igb0 is already configured).
3. Your network switch must properly support lagg

I've never seen an IPMI device that is an Intel card (which em0 is Intel). IPMI is NOT made available to the OS as part of its network ports. That is, you cannot send system data out of the IPMI port. So any concern you have with em0 being your IPMI card is unfounded.

Assuming everything "supports" lagg, I bet if you delete your igb0 configuration (which will default to dhcp at that point) will let you create the lagg without problems. ;)

On Supermicro X10s, the IPMI's default is to use its own management port, failing over to the Intel i210 (or one of them, on boards with several of them) if its controller doesn't work. This would be transparent to the OS, so it's not the issue here, most likely.

em0 is the only NIC being detected, so I'd venture a guess the other one is either disabled, broken or under the influence of a driver problem, like the one we discussed a few days ago. We do have an i217LM plus an i120AT on this board again...

 

[silbro]

Thanks a lot for the replys!

@cyberjock
This did the trick ;D I had to remove igb0 first and then em0. I scanned the network to see the new IP and then I created a failover link aggregation and configured that:

I even tested if the network cards would switch when I unplugged 1 of them. This worked, except 1 strange behavior:

when I unplugged eth0 it had a short delay which is normal (same behavior when I unplugged eth1.):

When I plugged back in eth1 it keeps working. When I plug back in eth0 it just takes longer to work normally again:

After theses time outs its stable again.

What I find strange is that on the info screen it shows em having an IP:

I can't ping that IP nor can I ping that subnet from the freenas.

 

[cyberjock]

Yep.. sounds like your hardware doesn't support lagg properly.

See #3 above. ;)

 

[silbro]

true :) thx a lot for the help! I guess this thread can be marked as solved

 

[pjc]

What kind of switch support is required for "simple" failover?

Also, what MAC address is sent out for a lagg DHCP request? The address of the first NIC in the group?

 

[Mlovelace]

You are trying to lagg two different ethernet controllers. Your Motherboard has Dual Gigabit Ethernet LAN ports: 1x Intel i217LM & 1x Intel i210AT which use two different drivers, hence the igb0 and em0. That might be the problem, I've never tried to lagg two different controllers.

EDIT: The i217LM white sheet shows support for teaming but I don't see that on the i210AT. Anyone know if you can team an i210AT?

 

[Mlovelace]

What kind of switch support is required for "simple" failover?

Also, what MAC address is sent out for a lagg DHCP request? The address of the first NIC in the group?

You have to have a switch that supports IEEE 802.3ad (IEEE 802.1AX)
http://www.newegg.com/Product/Product.aspx?Item=N82E16833150087 would work, I haven't shopped managed home/small business switches in a long time.

 

[pjc]

You have to have a switch that supports IEEE 802.3ad (IEEE 802.1AX)

The specs for my switch say "Port Trunking - Manual as per IEEE802.3ad Link Aggregation".

Will that work?

 

[Mlovelace]

Port trunking... is that a cisco switch? what's the model number?

 

[pjc]

Netgear GS724Tv2

 

[Mlovelace]

That should support link aggregation.