TrueNAS Mini running Scale 22.12.0 cannot create link aggregation with Unifi 10 GbE switch

[Lee DeCarlo]

I have a TrueNAS Mini running Scale 22.12.0 and cannot configure link aggregation with its two built-in 10 GbE ports and a Unifi 8 port USW-Aggregation switch. I have been searching for weeks on how to resolve this issue without any results. The NAS connects at 10 GbE on both ports without aggregation configured however attempting to configure the bond interface using LACP while also configuring the two ports on the switch for aggregation fails the connection test. I have attempted to setup the NAS networking using both the GUI and via the command line over IPMI. Note, I have a proxmox server using link aggregation on this same switch without issue. Any suggestions? Thanks

 

[morganL]

Please be more specific about what step you have taken and what the result was. You should include IP addressing info.

For example, did you get to confirm the network change in the UI?

 

[Lee DeCarlo]

Here are the steps I followed from my last attempt:
Before starting, the 8 port Unifi aggregation switch ports (#3 & 4) were working at 10Gbps and the TrueNAS Scale server was assigned with two static /24 addresses (10.0.0.3/24 & 10.0.0.4/24). These IPs are set to fixed in the Unifi UDM GUI for the client. They are configured the same way as the ports for my Proxmox server that is working with two bonded 10Gbps ports.

Last attempt:
- Configured the 8 port Unifi aggregation switch operation to "Aggregate" and assigned the aggregate ports for #3 & 4. The Link speed on the switch ports is manually set to 10Gbps FDX. As expected I would lose the connection to the switch so I would use the IPMI connection to the Mini to configure the connection on the server.

- On the TrueNAS server I would select "Configure network interfaces", then create a new network interface with the following settings:

type: Link_Aggregation
name: bond0
description: empty string
ipv4_dhcp: No
ipv6_auto: No
aliases: empty list
lag_protocol: LACP
lag_ports: eno1, eno2
xmit_hash_policy: None
lacpdu_rate: Slow
mtu: default

I ran the connection test but it was unreachable and would return to the previous state after 60 seconds.

 

[danb35]

Your settings look right, though I wonder about eno1 and eno2 being 10G interfaces. Did you delete the configuration on those two interfaces before configuring the LAGG?

 

[Lee DeCarlo]

Sorry, I should have mentioned the entire model name, it is a Mini X+ which comes with dual 10G interfaces standard. When I test the network changes it appears that the NIC configurations are removed, do I need to actually delete them and them and add the LAGG in one change? I see the IPv4 address go away on the IPMI KVM when testing so I was assuming they were removed when the LAGG was being deployed.

 

[Lee DeCarlo]

@danb35 I attempted to first remove the two interfaces first via the CLI and then add the LAGG but it made no difference. There is very little I can find for Scale (I'm running) versus Core to setup this up. The only difference I can see is the name of the interface needs to be bond0.

 

[paddy01]

As a final step, did you actually assign an IP address to bond0?

 

[IronDuke]

Here are the steps I followed from my last attempt:
Before starting, the 8 port Unifi aggregation switch ports (#3 & 4) were working at 10Gbps and the TrueNAS Scale server was assigned with two static /24 addresses (10.0.0.3/24 & 10.0.0.4/24). These IPs are set to fixed in the Unifi UDM GUI for the client. They are configured the same way as the ports for my Proxmox server that is working with two bonded 10Gbps ports.

Last attempt:
- Configured the 8 port Unifi aggregation switch operation to "Aggregate" and assigned the aggregate ports for #3 & 4. The Link speed on the switch ports is manually set to 10Gbps FDX. As expected I would lose the connection to the switch so I would use the IPMI connection to the Mini to configure the connection on the server.

- On the TrueNAS server I would select "Configure network interfaces", then create a new network interface with the following settings:

type: Link_Aggregation
name: bond0
description: empty string
ipv4_dhcp: No
ipv6_auto: No
aliases: empty list
lag_protocol: LACP
lag_ports: eno1, eno2
xmit_hash_policy: None
lacpdu_rate: Slow
mtu: default

I ran the connection test but it was unreachable and would return to the previous state after 60 seconds.

I don't know if this is helpful, but I was configuring two 1G ports against my switch (although not a Ubiquiti switch) in LACP mode, and I found I had to set the LACPDU rate to fast for the switch to like it. Also in the config above, "aliases" is "empty list". Looks like there is no IP address set, because that would show under aliases (unless it was DHCP, but that's "no" too).

In testing, I found there was no real gain, because each flow would use only one interface, not both simultaneously. You could have multiple clients using the LAG simultaneously, but only a single interface per flow. I can saturate a 1G, but not a 10G with my traffic, so I just moved to a single 10G. Sounds like you need the LAGged 10Gs though.