FreeNAS as guest in VMWare ESXi: What SATA Controller works???

[mattlach]

Hey all,

I've been researching, looking at VMWares compatibility list, looking at Whitebox compatibility lists (which all seem outdated and reference ESXi 3.5 and 4) and am ready to rip my hair out.

Here's my issue.

I need to run FreeNAS as a guest in ESXi 5.1

1.) My plan was to pass through the on board AMD750 storage controller to the guest (which reportedly works) but then I need another SATA controller for ESXi boot and datastores.

2.) Or I could pass through a SATA Controller card to the guest, and use the on board AMD750 for host drives.

Can anyone recommend me a low cost SATA controller card with 2+ ports known to work in ESXi 5.1 or a relatively low cost SATA 6 port controller card known to work with passthrough?

I'm having issues with my external RAID enclosure, and have bought all the parts to build host my FreeNAS guest, but I am stuck on the SATA controller card.

I'd appreciate any suggestions!

 

[mattlach]

On another forum I frequent I have received several recommendations for going with a IBM M1015 SAS controller, flashing it with an LSI IT firmware so that it becomes a simple JBOD controller, and then using it with 4-sata breakout cables for a total of 8 drives per port.

I'd still appreciate feedback from these forums on how well that would work with FreeNAS.

 

[pbucher]

I'm using the LSI version of that card for exactly this purpose and it works great. You need a few tricks to make it work.

The big problem is under ESXi it causes FreeBSD to hang when it boots. I've been trying various things for the past several months and here is my current solution which I've only had in production for a few days but it seems to be working better then any other things I've tried.

Add the following 2 Tunable entries under FreeNAS:

Variable - Value - Comment
hw.mps.disable_msix - 1 - Make mps driver not hang during boot
hw.intr_storm_threshold - 4000 - Push out threshold for LSI card
​

The interrupt storm one may or may not be needed, you won't have any issue with it if you use a single CPU. May setting is based on for CPUs allocated for the VM. My interrupt storm issues have greatly cleared up since I started disabling only MSIX interrupts for the MPS driver and left the plain MSI ones enabled. Most google results point you towards disabling both to get it to boot. Also the number of drives attached and traffic volumes will have some play into it all.

Anyways I've been running my home setup with a LSI card and a sata breakout cable without issue for many months now. My work boxes all have large SAS enclosures attached to them with SAS drives in them.

Anyways I'd also recommend looking at LSI's list of "certified" hard drives that work with their card. I've found the WD RE3 & RE4 SATA drives to be excellent performers for this sort of setup.

 

[pbucher]

Opps forgot the most important piece.

Install FreeNAS and add those entries before you add the LSI card to the VM and when you install an upgrade. Do the following:

1. Shutdown VM and remove LSI card from the VM.
2. Boot up VM and install upgrade - via CD Image or GUI upgrade.
3. Let it reboot and upgrade the database and then it reboots again.
4. Shutdown VM and add LSI card back to the VM.
5. Power Up and everything should be good to go.

 

[mattlach]

Thank you very much for these pointers!

I'm sure they will be very useful when I get the hardware and start installing it.

Great info as always.

--Matt

 

[mattlach]

Thanks again for all your help.

Just set it up today, and your tunables recommendations did the trick.

I had initially forgot about these, and went troubleshooting on my own, and found instructions on some config strings to add to loader.conf, but I couldn't get this to work, as the location loader.conf was in was on a write only file system. I'd edit fstab to make it rw instead of ro, and then when I rebooted it was right back to ro again...

Then I remembered this thread, and tried your suggestion instead. Worked like a charm.

Now speedtesting :)