It works BUT it doesn't like the adapter or M2 ssd drive and turns the fans on at high speed and becomes very noisy.
So I guess in a way is more correct to say that it doesn't work
I just purchased a near identical rig, R730xd 16+2 LFF with the H730 mini, so this is all very familiar. I installed the same NVMe drive in a PCIe adapter. The ramped up fans when using an unsupported device is expected behavior. I found a video that walked me through setting the fan speed to a static value manually to quiet them down. I set them to 60%, not great but quieter than 100% for sure. There are also scripts that will take that a step further and allow the speed to increase when a certain temp is reached rather than remain at the static value (percentage). I have not gotten there yet but some googling should point you there.
The Dell Internal dual SS module (IDSDM) can be found on eBay relatively inexpensive as can Dell 16GB SD cards. It’s plug and play as it’s Dell and will be visible in the bios. This is the route I went to boot ESXi, with the cards set to mirrored in the bios. The installation was painless and Dell has a video on the whole process, including the ESXi install.
The reason I installed the Samsung 970 m.2 was to confirm my hope that the drives would show hen the controller was attached to the vm. My intent was (and still is) to create the FreeNAS vm on the SSDs in the flex bay, but because they are attached to the controller that was passed through and not visible to ESXi, I needed somewhere else to install it and I already had that drive handy.
I was able to create a Datastore with the NVMe drive and create the FreeNAS vm on it. I attached the passed through controller (I had already swapped out the H730 mini for an HBA330 mini mono (which I just read here is not ideal, bummer) when I couldn’t see the drives in ESXi after passing it through (because I too was new and confused). After installing FreeNAS, I confirmed all the drives were visible. I did have to open the GUI in a different browser to get the drive info to populate. At first it only showed the serial numbers but no sizes or anything else. After the info populated, it was visible in all other browsers.
Because I ultimately want FreeNAS to run on the 2 rear flex bay SSD drives (mirrored), I purchased another H330 (card, not mini mono). Unfortunately, the description in the listing was wrong and I received the HBA330 instead, and I want the drives presented to ESXi as a mirror so I need the RAID. I’m waiting on the correct one now.
In the meantime, just to make sure everything will eventually work, I installed the HBA card in an open riser slot. I disconnected the cable that connects the rear backplane to the front backplane at the front end and rerouted it to the HBA card. This causes a warning at pre-boot that that the cable is disconnected, two actually - one for each end. It can be bipassed by pressing F2 to enter setup from said warning screen and then just exiting setup. It will continue to boot to ESXi just fine, and the drives are visible and available in ESXi. It will also cause the amber status light to stay on, but from what I have read and noticed, it has no other effect. I did try removing the sensor cable from the rear backplane in an attempt to circumvent the error, but it had no effect. So I’ll live with the error and no blue light even though it bugs me.
I now have no reason to believe my original configuration plan won’t work when the correct card arrives later this week.
This will be an AIO server with other VMs (Plex, lab, etc. in ESXi, not FreeNAS). I think I might keep the NVMe drive in it and use it as the OS drive for the other VMs. I will have to implement the fan control script to make that a viable option.
I know this was lengthy, but I wanted to share the whole process because it was so recent and similar. I know it would have helped me a lot. I realize you may have already found your way in the last few weeks. If not, hopefully some of my experience will be useful to you. Hopefully it will help at least someone.