Multiple HP P2000 enclosures and FreeNAS

[joriz]

Hello,

I want to repurpose some hardware which exists out of 4 HP P2000 enclosures with 12 x 3TB disks each and 1 HP Proliant DL380 G7 as FreeNAS host.
The purpose of this setup is to temporary store data before it moves to the backup system. IOPS are not important, redundancy and capacity are.
There will be no deduplication or any drives for caching, just the disks in the enclosure.

The specifications:
- HP P2000 with dual sas controller (non raid)
- 12 x 3TB in each enclosure
- Dual Intel Xeon E5620
- 48GB ECC DDR3
- 2 x dualport SAS HBA (LSI 9200-8e)

As IOPS are not important i want to use VDEVS of 12 disks each and raidz2.
Also the enclosures are connected such that multipath can be used over 2 HBA's.

Questions i have:
- As the dataset spans all 4 enclosures, all the data will be gone when 1 enclosure fails.
- I currently have 1 vdev per enclosure. Would it be better to span 1 vdev over all 4 enclosures for additional redundancy/performance ? I think not...

Please share your opinion.

 

[Elliot Dierksen]

That is too many drives per vdev based on my understanding of I/O. I would say 2 x 6 drive RAIDZ2 vdevs per enclosure. I can't speak to the P2000 enclosures specifically, but I have done pretty close to the same thing with a D2600 and a D2700 external enclosure attached to an LSI 9207-8e. That has worked well for me. It is not my primary FreeNAS, but a backup. Still no issue so far with that.

 

[joriz]

Thanks Elliot. 2 x 6 drive in raidz2 would be better yes.

Did your enclosures also had dual controllers? I'm wondering if multipath in Freenas was working out of the box for you.
My test Freenas install detects 1 physical disk twice when creating the pool and the multipath option is not available in the menu.

 

[Johnny Fartpants]

I've done 10 disk Z2s and currently, 15 disk Z3s so you are not a million miles away with 12 disk Z2s particularly as you mention its a temporary store.

Spacing vdevs across boxes for extra resilience isn't a stupid idea but for it to be worthwhile you would have to size the vdevs differently so either 4 disk Z2 (one disk per JBOD), 8 disk Z2 (two disks per JBOD) 12 disk Z3 (three disks per JOBD). However, saying all that and again based on this being temporary storage it's almost defo not worth the hassle.

A pool per JBOD is also not a bad idea depending on how you want to use the space. Naturally, you lose the ability to share space across the JBODs however gain the ability to simply export any of the JBODs and connect them to a separate head and namespace should you ever require at a later date.

I run multipath on most of my systems and it works well out of the box however I change the naming scheme to help me identify disks but I can't speak for 11.3 as my systems are still on 11.1 or 11.2.

 

[Johnny Fartpants]

As the dataset spans all 4 enclosures, all the data will be gone when 1 enclosure fails

Yes in your current suggested config.

 

[Elliot Dierksen]

Did your enclosures also had dual controllers? I'm wondering if multipath in Freenas was working out of the box for you.
My test Freenas install detects 1 physical disk twice when creating the pool and the multipath option is not available in the menu.

Yes, it did. Both controllers come back to a single 9207-8e. It looks like you have 2 HBA's which is different from my configuration.