LAG: 2 Debian Server <-> FreeNAS: better performance possible?

[fips]

Hey!

My Setup:
2x Debian Server with each 3 Intel NIC's
1 Cisco managed Switch with LACP
1 FreeNAS with Quad Intel NIC

I set up bonding on both servers and on that Freenas.
Either i enabled LACP and created LAG for the right ports of that Server.

So far i only have single gigabit performance...
I know that LACP doesn't bring more performance on a single TCP connection, but if BOTH server access the FreeNAS?
Under those circumstance i really should see a difference.

I really can't see an error, maybe you can bring me some "ideas" where i could have a look on...
Maybe together we can find my mistakes :D

THANKS in advanced!

Steve

 

[cyberjock]

No. You are being sold on what isn't real.

LACP works great for a high number of connections. LACP will split the connections (keyword: connections) approximately 50/50(keyword: approximately).

The *load* will only be split if the users happen to split the load across the connections (very unlikely). It's also approximately 50/50 because the algorithm that decides which connection goes to which port is basically like a random number generator. Is it possible that you'll have 50 connections all on the same NIC? Yes, but it's not likely. Now what if it's just 2 connections? Now you have a 50/50 chance of seeing no benefit whatsoever.

So unless you're about to tell me that you plan to have dozens of connections then you aren't going to see the performance boost you are wanting. And you *definitely* aren't going to see it from one machine to another.

 

[fips]

Hmm...
But how does somebody like here come to his result?
https://onapp.zendesk.com/entries/3...ode-4-traffic-goes-through-1-NIC-instead-of-4

 

[cyberjock]

That is someone that hasn't:

1. Setup LACP properly on his server
2. Setup LACP properly on his network switch
3. Doesn't have a compatible network switch
4. Doesn't have a compatible network card
5. Some combination of the above.

Do you really want *all* of your traffic to go through one NIC like that guy had? If so that's easy.. get rid of the extra NICs. ;)

 

[fips]

Now i confused...
This guy has 4 NIC and can manage the performance of 4 times gigabit traffic only by changing the hash policy.

OK new try:
Could ANYBODY ever increase his bandwidth over lacp or other mode's except install 10GBit NIC?

 

[cyberjock]

Not from one single machine to any other single machine. That's where 10Gb is better.

If you are using iSCSI you can do MPIO and have 1Gbx(however many links), but you are limited to iSCSI for that.

 

[c32767a]

Hey!



So far i only have single gigabit performance...
I know that LACP doesn't bring more performance on a single TCP connection, but if BOTH server access the FreeNAS?
Under those circumstance i really should see a difference.

I really can't see an error, maybe you can bring me some "ideas" where i could have a look on...
Maybe together we can find my mistakes :D


Steve

This is expected. LACP divides traffic between links based on a hashing algorithm.
Some devices have configurable hashing, but in nearly all cases, the input to the hashing algorithm is the ethernet MAC address and/or IP address of the packets.
This has the effect of putting all traffic between 2 hosts on the same link of an LACP channel, even if multiple links are available.

LACP is only valuable when you have enough different senders and receivers that the hashing algorithms can distribute the traffic across all the links in the channel.

If you only have 1 or 2 devices talking across the LACP link, it is a much better decision to switch to a faster link speed (e.g. 10G) rather than aggregate more slower links.

Some OSs have hacks to distribute traffic across multiple links (e.g. MPIO, Microsoft NLB, SMB4, etc) but those are OS and implementation specific and probably won't work with FreeNAS unless they are explicitly supported by FreeBSD and documented in the FreeNAS manual. They also might not work with your network infrastructure.

 

[akc360]

Just create 2 Vlan on your switch, One for each deb server. then use different subnets for each server to NAS connection(this will let you pull 2x 1Gb from Freenas). you also have enough nics to have a common vlan connection to allow inter computer/internet communication.

 

[titan_rw]

I got into this at work recently.

These are all windows boxes, but running dual gigabit nics configured in LACP.

Our core switches (cisco 3750's) were all set to load balance based on source mac address, which doesn't work so well when the majority of traffic comes from a router. The traffic will all have the same source mac, the routers. I changed the switches to load balance based on src-dst-ip, which if I remember right is the source ip xor'd with the destination ip. This is how traffic is load balanced going into the server.

For traffic coming FROM the server, I set HP NIC team utility to "dynamic load balance" which seems to work the best. The server picks the lowest utilized nic when it needs to decide which port to send traffic on.

After setting up the correct etherchannel's on the switch side, I verified with iperf from two separate physical machines. Once again, traffic from box A to box B will only ever do 1 gig. But if box A is sending traffic to both box B and C at the same time, I've seen a total of 1.9 gigabit over the NIC team. This has real world results when using WDS to image workstations. Theoretically we can have two machines imaging at full gigabit each.

But with two different server's connecting to freenas, I'd just set them on separate vlans and split / dedicate traffic that way. As menioned above, that will guarantee each server full gigabit to freenas.