iSCSI MPIO for ESXi 5.5 and NTFS Volumes

[Ronan McGurn]

Hello All,

I hope I'm not beating a dead horse, but I've been reading up on the iSCSI implementation using FreeNAS and would like some clarification/assistance if possible. I'll summarize what I',m trying to do, and I'm open to any suggestions and will be happy to supply additional information if desired/required.

Hardware:
Catalyst 3560-48 10/100/1000 switch
Cisco 2951 ISR G2 router (using subinterfaces for vlans on 3560, gi0/2.1, 2.2, 2.3, etc, doing inter-vlan routing)
ASA 5520 Firewall (most likely irrelevant to this since I'm not planning on accessing storage from WAN at this time)

Dell PowerEdge 2900 2x Dual-Core Xeon @ 2.66GHz, 16GB RAM, 2x Broadcom on-board GbE, (bce0,1) 3x Intel Pro1000/MT Dual-port GbE NIC (6x em interfaces, em0,1,2,3,4,5,), PERC5/i controlling 5x 1TB SATA and 3x 300GB SAS. Arrays have been created on the PERC, not using ZFS. Not sure if this is best or not, but not my issue here.

4x Dell PE 1950 ESXi 5.5 hosts each with 2x Intel GbE NICs dedicated to iSCSI (which has been configured using a 1:1 ratio of vmkernel to vmnic and bound properly to iSCSI HBA using round-robin MPIO on vmware side.

So, I'd like my FreeNAS box to have 6Gbps throughput using MPIO, thus the 6x GbE interfaces dedicated for such, the on-board broadcoms are for management only.

I understand that I need a separate non-overlapping subnet for each interface i plan to use with FreeNAS for iSCSI MPIO, although apparently in earlier version (prior to 9.1 which I'm using) this was doable using multiple IPs in the same subnet. My first question, should I create a separate vlan on my switch for each iscsi NIC, such as:

em0: 10.1.1.10 > vlan300
em1: 10.1.2.10 > vlan310
em2: 10.1.3.10 > vlan320
em3: 10.1.4.10 > vlan330
em4: 10.1.5.10 > vlan340
em5: 10.1.6.10 > vlan350

Next would be switch config, if I setup these switch ports as trunkports, how do I assign vlan tagging in FreeNAS? I tried creating a vlan (vlan300) and using parent interface em0, used "300" for vlan number, and then in "Interfaces" configured IP as 10.1.1.10/24 (I know I don't need a /24 but this is testing currently, just keeping it simple currently). I configured the switch interfaces as trunkports, but was unable to ping the IP I assigned. I was able to ping an SVI I created in the switch in the same vlan however so vlan is not issue. Possibly I'm missing on how to configure FreeNAS interface to work with a trunkport? If I configure the switchport as an access port in the vlan, i.e, vlan300, then I can ping it fine. So that's one issue.

If I'm correct from what I'm reading, I have to create an interface in each vlan like above for iSCSI MPIO? Then I create an iSCSI Portal to access these? Now my next question, my initiators network config, I currently have 1 vlan for iSCSI traffic (250, 172.16.1.0/24) and all my vmkernal NICs are in that subnet, and I had previously had an LACP EtherChannel connecting the FreeNAS box to the network, using the lagg interface, which while it did work, I don't believe I was getting performance close to what I ought to, mainly because I was using LACP as opposed to MPIO (mistake I know, I went the "easy route" because I was in a hurry. Do I keep my initiators on their current vlan (250), and put an iSCSI "portal" IP Address somewhere in that vlan as well, and then use the "Portal" to access the targets using the other interfaces? This is where I' not sure. Any thoughts, welcome. Thanks in advance.

 

[dlavigne]

Did you ever figure this out?

 

[Ronan McGurn]

No. Unfortunately.

 

[eraser]

4x Dell PE 1950 ESXi 5.5 hosts each with 2x Intel GbE NICs dedicated to iSCSI (which has been configured using a 1:1 ratio of vmkernel to vmnic and bound properly to iSCSI HBA using round-robin MPIO on vmware side.

Can you clarify something for me? In each ESXi host are you using a physical iSCSI HBA card or are you using 2 x Intel GbE NICs? (or both in some strange combination)?

 

[eraser]

One more question -- your post title mentions NTFS.

Typically when iSCSI storage is presented to an ESXi Host it is then formatted using the VMFS filesystem and used as a datastore to hold Guest VM files (config files, .vmdk virtual disks, etc). This is the configuration that can show improved performance when MPIO is configured at the ESXi-level.

However, if your goal is present iSCSI storage directly into a Windows VM so that it can format it as NTFS, then you will want to pretend that the Windows VM is physical and configure MPIO inside the Windows VM itself.

I think the first method is most likely what you want, but thought I'd double-check first.

 

[c32767a]

Hello All,


Next would be switch config, if I setup these switch ports as trunkports, how do I assign vlan tagging in FreeNAS? I tried creating a vlan (vlan300) and using parent interface em0, used "300" for vlan number, and then in "Interfaces" configured IP as 10.1.1.10/24 (I know I don't need a /24 but this is testing currently, just keeping it simple currently). I configured the switch interfaces as trunkports, but was unable to ping the IP I assigned. I was able to ping an SVI I created in the switch in the same vlan however so vlan is not issue. Possibly I'm missing on how to configure FreeNAS interface to work with a trunkport? If I configure the switchport as an access port in the vlan, i.e, vlan300, then I can ping it fine. So that's one issue.

If I'm correct from what I'm reading, I have to create an interface in each vlan like above for iSCSI MPIO? Then I create an iSCSI Portal to access these? Now my next question, my initiators network config, I currently have 1 vlan for iSCSI traffic (250, 172.16.1.0/24) and all my vmkernal NICs are in that subnet, and I had previously had an LACP EtherChannel connecting the FreeNAS box to the network, using the lagg interface, which while it did work, I don't believe I was getting performance close to what I ought to, mainly because I was using LACP as opposed to MPIO (mistake I know, I went the "easy route" because I was in a hurry. Do I keep my initiators on their current vlan (250), and put an iSCSI "portal" IP Address somewhere in that vlan as well, and then use the "Portal" to access the targets using the other interfaces? This is where I' not sure. Any thoughts, welcome. Thanks in advance.

I think you're making this overly complex. You have 6 discrete switch ports on the NAS, and 2 on each of the ESX boxes. They will all be connected to the same switch. The biggest problem you're going to have is that you have 8 ESX ports and 6 NAS ports. There's really no balanced way to merge 6 into 8.
You are either going to end up with oversubscription somewhere, the question is, where? do you just want to dedicate 1 NAS port to each ESX machine and do 2:1?

Am I missing something, or is that the physical layout? If that's what you want to do, I can walk you through how I would provision the vlans and ESX machines.