2 X NIC's, (LAGG), Load-balance or crossover cable FreeNAS to VMWare ESXi5 (iSCSI)

[leenux_tux]

Hello Forum,

I currently use FreeNAS to provide a number of services for my home and home office/work environment, backups, serving up media content around the house and as an iSCSI target for my ESXI 5 server.

I have been going through a number of improvements over the last 6 months or so and making additions and modifications where possible, installing 8GB or RAM even when the motherboard documentation stated only 4GB was supported. Adding a dedicated vSwitch in VMWare for iSCSI traffic (my VMWAre box has two NICS, the vSwitch is configured for the on-board NIC). Installing a network switch and replacing all my 1TB drives with 2TB ones so I have more space (I never thought I would need more than 2TB !!).

What I am thinking of doing now is add another NIC to my FreeNAS box, which is a tad awkward (physically) but can be done using a PCI flexi-extender, however, I'm interested in getting some feedback from the forums experiences with this, I'm looking for the best option for speed as far as the VMWare to FreeNAS "iSCSI" connection is concerned.

Do I use Link Aggregation and configure Load Balancing ? I have read ("http://doc.freenas.org/index.php/Link_Aggregations") that you need a switch that will support "IEEE 802.3ad static link aggregation" for this to work, this could mean I have to buy a new switch.

Or do I get a crossover Ethernet cable and connect the two systems together directly. Will that give me the best throughput I can get with the limited hardware I have i.e small cost/big throughput improvement?

Thanks

 

[cyberjock]

I just tried to use a 20cm flex extender for PCI on Saturday. Card would not work with the extender.

LAGG and LB really is a decision you have to make for your own configuration. It's really hard to give an opinion without a very thorough understanding of your LAN setup and users.

At home I use 2 IPs for my server. One is for the rest of the house and one is for me. I map them with the IP to force the connections. Since I'll be the only one transferring GBs and GBs of data there's no reason to share with the rest of the house.

 

[ProtoSD]

FreeNAS does not do load balancing, there are a couple of threads where it was discussed.

 

[leenux_tux]

noobsauce80

I will give the extender a try anyway and see what happens.

This is a home-home/office system so is 95% used by myself, with the odd backup from a one other laptop in the house. There are only ever a maximum of 4 connections to the FreeNAS box at any one time, including the ESXi system. In fact, I sometimes have more virtual machines running and "virtually" connected to FreeNAS (via iSCSI obviously) than I have physical systems connected. I have been finding the iSCSI capabilities of FreeNAS very useful in regards to my ESXi testbed so was interested in finding out if the dedicated Ethernet cable was a good/viable solution.

 

[leenux_tux]

Strange.... The documentation I have read seems to say otherwise (http://doc.freenas.org/index.php/Link_Aggregations)

There is a section that refers to "Load Balancing" and states the following (quote) "balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. The hash includes the Ethernet source and destination address, VLAN tag (if available), and IP source and destination addresses. This is a static setup and does not negotiate aggregation with the peer or exchange frames to monitor the link. Requires a switch which supports IEEE 802.3ad static link aggregation." (unquote)

It does specifically state that the switch should support a specific "IEEE" standard though.

Or is it a case of the documentation states one thing but in the "real world" it doesn't actually work ?

 

[jgreco]

If you only have a single ESXi host, this may be a pointless exercise (or require complex configuration: you can do multipath, but that's a pain to set up). iSCSI is implemented on top of TCP, so traffic to/from your ESXi host via that TCP connection is always going to select one port. It won't magically split across two. This is done because receiving TCP packets out-of-order can be very detrimental to the performance of TCP connections. To use both connections, you need to have two TCP connections *and* they need to happen to hash to different ports. This isn't a FreeNAS thing, it's just how link agg works. It is much more likely to work usefully if you have several ESXi hosts.

Link agg isn't impossible with iSCSI, but it is sufficiently annoying and painful that you really don't want to try until your iSCSI needs really do exceed 1Gbps per host.

This is actually what the FreeNAS docs above are trying to tell you. You'll find your ESXi docs say something similar.

 

[jgreco]

FreeNAS does not do load balancing, there are a couple of threads where it was discussed.

And that, by the way, is potentially misleading. A better statement: FreeNAS does not do per-packet load balancing across multiple ports for a single hash (think: a single TCP connection). Assuming FreeNAS uses the same link aggregation system FreeBSD does, link aggregation is certainly known to work fine in FreeBSD. You just have to understand what it will and won't do for you.

 

[cyberjock]

Um.. I hate to break it to you, but AFAIK there is no load balancing or LACP on a per-packet basis. I talked to 3 different companies, some do alternating connections and keep a list in the switch and others go by the remote network MAC address. The one company with the MAC address said that it is entirely possible if you have only a few machines that they will all use one port and the other will never be used.

You should read up more on LB and LACP, but for home use the cost is pretty much not justified except in specific scenarios.