zPool atop hardware RAID5

[ehowton]

Howdy! I’m running FreeNAS-11.3-U5 on a R730 with 7xSAS drives in a single vdev across a hardware RAID5 volume. As such, many of the zpool commands do not return detailed information, and individual disk options are unavailable. The server reports a disk error on one of the hard drives, but there is no functionality in FreeNAS to offline it of course. The hardware subsystem supports hot-swappable auto-reconfiguration, but I am unsure what replacing the disk will do to the zPool. Worst case scenario I can restore from backup but I was curious as to what to expect if anyone in the community knows. Thanks!
Required Specs:
Dell PowerEdge R730 Gen13
32-thread Xeon E5-2640v3 @ 2.60GHz
16GB PC4-2400T ECC
7x10k 146GB SAS (Internal USB boot)
PERC H730P RAID controller
(Intel?) 10GB quad-port HBA

 

[jgreco]

Expect that it will either fail or work, possibly with some data loss.

Please do review the following link;

https://www.truenas.com/community/r...bas-and-why-cant-i-use-a-raid-controller.139/

 

[ehowton]

Appreciate the reply (I too expected it would either fail or work lol). I do believe I'll use this opportunity to test passthrough in the BIOS to present the individual drives to FreeNAS. Thanks again!

 

[jgreco]

On re-reading that, just to be clear, it may:

1) fail
2) work
3) work with some data loss

We can't really know for certain, for all the reasons linked in the linked article, plus, you haven't told us what kind of RAID controller it is. I should probably add that it would be a good idea to evacuate all the data from the pool to someplace safe, and then rebuild the pool with a compatible HBA crossflashed to IT mode, which will give you a much more pleasant and reliable experience.

 

[ehowton]

I apologize - the forum rules seemed to infer the preference of obfuscating that nugget behind a spoiler tag. Its a PERC H730P, which does support IT mode. In all likelihood I will do just as you suggest :)

 

[jgreco]

No, the PERC H730P does not actually support IT mode. It has a so-called "HBA" capability which is actually just to disable the RAID functionality, but it continues to use the same firmware and host device driver, so it is merely a lobotomized RAID card. This is what Point 5) in the article I linked to is all about.

The point of our insistence on a true HBA is because we really need something that won't buckle under the load.

 

[ehowton]

Thanks for the input. I was certain I'd triple-checked the HCL, and the RAID card was the entire reason I'd put FreeNAS on the R730. I appreciate your time today.

 

[jgreco]

If you checked the Free__BSD__ HCL, I'm sure it's on there. The problem is that ZFS demands a whole new level of a particular kind of correctness, so while it may work with Free__BSD__ in a conventional server role, and I'll bet it's an awesome controller for all sorts of hardware RAID-y feature goodness there, unfortunately that doesn't really make it Free__NAS__ compatible. But I already linked to the article that explains why.

Overall, I hate LSI RAID cards (the H730P being a 3108 variant), but I use them because everything else is significantly worse. I'm experimenting with a Dell PERC H740P in one of our ESXi servers, and despite the incredibly shallow 32 queue depth, the thing will stampede several gigabytes of I/O into cache before slowing down to the ~500MBytes/sec that the backing SSD's support.