Multipath not working correctly

[Pascal Robert]

Hi,

I'm trying out FreeNAS 9.3 with Citrix XenServer 6.2. To get the best performance, my iSCSI target on FreeNAS is configured with 3 IPs so that I can use multipathing (MPIO). Same thing for my XenServer host, I have a quad-port Intel i350 network card that is connected to the FreeNAS host (SuperMicro server with a Atom processor and two SSDs).

Everything works fine, except the performance. The trafic only goes to one interface, the two other interfaces don't have any trafic. XenServer shows that 3 iSCSI sessions are active, and iscsiadm shows the 3 IPs.

For those using XenServer with FreeNAS and iSCSI, what kind of setup do you have?

 

[HoneyBadger]

On FreeNAS:

Are your iSCSI settings correct to add all three IPs to the portal? All three should be tied to the same portal.
Are all three IPs on non-overlapping subnets? You will only see outbound traffic on one IP per subnet from the FreeNAS machine.

On Xen:
Is iSCSI configured to use round-robin pathing, or fixed/MRU?

On nothing in particular:
Odd numbers make me uncomfortable. Any reason you aren't at two or four interfaces?

 

[Pascal Robert]

On FreeNAS:
Are your iSCSI settings correct to add all three IPs to the portal? All three should be tied to the same portal.

Yes, they are tied to the same portal.

Are all three IPs on non-overlapping subnets? You will only see outbound traffic on one IP per subnet from the FreeNAS machine.

All different subnets. 192.168.5.3 for first interface, 192.168.6.3 for second interface, 192.168.7.3 for third. Netmask of 255.255.255.0 for each.

On Xen:
Is iSCSI configured to use round-robin pathing, or fixed/MRU?

I don't see that option in XenCenter, but in /etc/multipath.conf, I see:

#path_selector"round-robin 0"

On nothing in particular:
Odd numbers make me uncomfortable. Any reason you aren't at two or four interfaces?

The fourth interface is for management.

 

[HoneyBadger]

I don't see that option in XenCenter, but in /etc/multipath.conf, I see:

Code:

#path_selector"round-robin 0"

Not a big Xen guy, but that # seems to be a comment tag, delete it so the line is just

Code:

path_selector "round-robin 0"

Also see this thread for other settings that worked for someone on Xen 6.0.2 - rr_min_io 1 should force it to hop every I/O for better balancing.

The fourth interface is for management.

Not a big deal. Three is just, well, it's weird.