Need help on ZIL drives partitioning & multiple vDevs for VMware/ESXi LUNs

[Redglobal]

Hi,

I'm new to FreeNAS/Linux and need some help on creating multiple vDevs with mirrored SSD ZIL. I'm trying to achieve the best possible performance for my VMware hosts in a lab environment from the following Hardware:

System & Network info:
HP ML 150 G6 with FreeNAS 9.10
1 x Intel Xeon with 48GB ECC Mem
2 x LSI 9211-8i, P19 IT (1 dedicated for SCSI & 1 dedicated for SATA drives)
Dedicated 3 x 1 Gb Intel NICs for iSCSI / NFS, will use multipath config with round-robin from ESXi side

Here are the details & targets of the drives:
7 x 900GB SCSI 10K for ESX/VMFS LUNs
4 x 5TB STAT 7.2K for NFS & CIFS, Jail etc.
2 x 240GB Intel SSDs for L2ARC (1 for SCSI & 1 for SATA pool)
2 x 120 Gb PNY SSDs for ZIL (Mirrored for ZIL)

I know MLC SSDs are not recommend for ZIL but I'd like to try these for now as the write speed, IOPS and MTBF is not too bad on these PNY drives, and my VMs are not critical and have a plan to back those up anyway.
My plan is to backup VMs on SATA LUNs via VMware snapshot feature of FreeNAS if possible on the same box. Or I'll run a FreeNAS VM guest with passthrough LSI & Intel NIC card on a separate host to replicate VM data.

Please let me know your design thoughts of ZFS volumes and recommendations. Also, please let me know the commands for under provisioning the ZIL drives/create partitions to 8GB on the 120GB/ZIL drives. I was also wondering if I could use any secondary partitions on the same 120GB drives for a second set of ZIL partitions for the SATA pool/vDev?

Any input and help would be greatly appreciated.

Best,

 

[depasseg]

I know MLC SSDs are not recommend for ZIL but I'd like to try these for now as the write speed, IOPS and MTBF is not too bad on these PNY drives, and my VMs are not critical and have a plan to back those up anyway.

Since I doubt the PNY's have power loss protection, and the performance isn't great, why not just disable sync writes? Your performance will be much better, and only marginally more risky.

I was also wondering if I could use any secondary partitions on the same 120GB drives for a second set of ZIL partitions for the SATA pool/vDev?

It's not advised, since the whole point of the SLOG is low-latency writes. But technically it is possible.

 

[gpsguy]

btw, you should be using the P20 firmware.

2 x LSI 9211-8i, P19 IT

 

[Redglobal]

btw, you should be using the P20 firmware.

I'm unable to locate the bin/rom file inside the P20 fw/bios zip package.. Can you guide me to the right direction please.

Also, is there any change logs available somewhere for p20? Just waned to check what are the major changes in fw version P20.

Best

 

[bigphil]

Just as a note for your iSCSI MPIO setup that you mentioned. Be sure that you DO NOT use iSCSI port binding on the VMware side. a lot of people do this incorrectly and it's is not the proper method for FreeNAS (or any NAS/SAN) with multiple interfaces on different subnets. Bind all IP's you're using for storage traffic to your portal group/target on FreeNAS and add each IP as a target on the VMware iSCSI config. You still need multiple vmkernal interfaces on VMware, just don't port bind them. this is best practice from the VMware guides.

 

[SweetAndLow]

I'm unable to locate the bin/rom file inside the P20 fw/bios zip package.. Can you guide me to the right direction please. View attachment 14919 View attachment 14920
Also, is there any change logs available somewhere for p20? Just waned to check what are the major changes in fw version P20.

Best

It's not really about the changes. It's the fact that freenas uses the p20 driver and firmware should always match the driver.

Sent from my Nexus 5X using Tapatalk

 

[Redglobal]

Just as a note for your iSCSI MPIO setup that you mentioned. Be sure that you DO NOT use iSCSI port binding on the VMware side. a lot of people do this incorrectly and it's is not the proper method for FreeNAS (or any NAS/SAN) with multiple interfaces on different subnets. Bind all IP's you're using for storage traffic to your portal group/target on FreeNAS and add each IP as a target on the VMware iSCSI config. You still need multiple vmkernal interfaces on VMware, just don't port bind them. this is best practice from the VMware guides.

So, do I need to setup the LAG in FreeNAS side? It's not letting me add multiple target IPs in the same subnet..

 

[gpsguy]

You probably have the wrong file. Use the one for version 20.00.07.00 at the bottom of this page: https://www.broadcom.com/products/storage/host-bus-adapters/sas-9211-8i#downloads

I'm unable to locate the bin/rom file inside the P20 fw/bios zip package.. Can you guide me to the right direction please

 

[Redglobal]

You probably have the wrong file. Use the one for version 20.00.07.00 at the bottom of this page: https://www.broadcom.com/products/storage/host-bus-adapters/sas-9211-8i#downloads

Thanks! Apparently, found it in their archive files for DOS/Win package.. BIN file is not present in their current files!

 

[bigphil]

So, do I need to setup the LAG in FreeNAS side? It's not letting me add multiple target IPs in the same subnet..

View attachment 14921

You wouldn't use LAG for iSCSI....only MPIO. LAG would be for NFS. I'm on vaca so can't look at the setup atm, but I was referring to multiple IP's on the iSCSI setup on FreeNAS. when you setup a portal, you can add multiple interfaces to it.