FreeNAS 11.2 and VMWare 6.7 performance issues.

[Fabiano de Oliveira Faria]

I use the FreeNAS suite of solutions with VMWare ESXi for many years and it works perfectly well in one environment. FreeNAS acts by serving disks through the iSCSI protocol for the VMWare ESXi environment, in production environment, for the company's virtual servers. In this environment, FreeNAS hardware is a DELL POWER T410 Edge with PERC H200, 32GB ram. The FreeNAS version is 9.10. I have more than 12 virtual servers distributed by 4 ESXi hosts and this environment presents no problems.

I'm setting up a new environment. This time FreeNAS hardware is a DELL POWER Edge T440 with PERC H330 controller. I use 6 Seagate IrowWolf 3TB each, and two 120GB SSDs each, 32GB ram. I also use 6 Gigabit NICs (2 + 4). The pool was built using RaidZ2 with 5 3TB disks and 1 spare 3TB disk. Already the SSD disks were 1 for cache and another for the FreeNAS system, respectively. The FreeNAS version is 11.2. This is the troubled environment.

The installation of FreeNAS went smoothly. I do not have any warning and it seems all right.

After creating the iSCSI stores and mapping them to VMWare, I installed a Windows Server 2008 R2 VM. There is no other VM configured in FreeNAS or ESXi. During Windows installation, I noticed that the time elapsed was too long. And if you try to install any application, it takes a long time to do so. But it is super fast to open the applications already installed.

Both the FreeNAS version and the VMWare version are newer in this environment compared to the previous environment. Hardware is also more powerful on both systems in the new environment.

I basically re-edited the same settings for iSCSI, network, etc. in this new environment.

Could someone help me with this?

I'm using google translator. Sorry for any grammar errors.

 

[RickH]

It's a little hard to tell from your description exactly what's going on, but it sounds like your write performance is much lower than you were expecting...

First of all, with 6 disks I would recommend 2 3-drive RaidZ2 vdevs for the best performance in ESXi. In my opinion, having a 'hot spare' drive is pointless and you would be better served by having all drives active....

Is your PERC H330 flashed to IT mode? I don't have experience with that exact card, but a quick Google search indicates that it's possible to get IT firmware on that card. If you've configured your system using the original Dell firmware you could be having some type of on-card write-back/write-through caching issue

It appears that you're not running a ZIL - if your pool is set to sync=always that could also account for the slow writes.
Run this command from the shell to see if sync is enabled for your pool:
zfs get sync <pool name>/<dataset>

NOTE: Despite the performance implications, setting sync=always is actually my recommended way to set up zfs with iscsi. If this is a lab/testing environment and your server is on a battery backup it's probably safe to turn it off.

 

[Fabiano de Oliveira Faria]

It's a little hard to tell from your description exactly what's going on, but it sounds like your write performance is much lower than you were expecting...

First of all, with 6 disks I would recommend 2 3-drive RaidZ2 vdevs for the best performance in ESXi. In my opinion, having a 'hot spare' drive is pointless and you would be better served by having all drives active....

Is your PERC H330 flashed to IT mode? I don't have experience with that exact card, but a quick Google search indicates that it's possible to get IT firmware on that card. If you've configured your system using the original Dell firmware you could be having some type of on-card write-back/write-through caching issue

It appears that you're not running a ZIL - if your pool is set to sync=always that could also account for the slow writes.
Run this command from the shell to see if sync is enabled for your pool:
zfs get sync <pool name>/<dataset>

NOTE: Despite the performance implications, setting sync=always is actually my recommended way to set up zfs with iscsi. If this is a lab/testing environment and your server is on a battery backup it's probably safe to turn it off.

I made some adjustments in the environment and so far, the results are satisfactory.
I changed the Raidz2 poll to RaidZ1.
I added the "mtu 9000" parameter in the link aggregation in both FreeNAS and ESXi.

I have not blinked the controller for IT mode because FreeNAS is not signaling any warnings. On the other server that uses PERC H200 I had to perform this procedure, but FreeNAS flagged warning until I did so.

Yes. Sync is active, but now as performance has improved a lot, I think it's better to keep it that way, because this environment will be production.

Thank you for your help!