30+ disk array advice

[Allan_Guy]

I have 30 2tb drives to put into an array. I have a asrock 2750 that has 12 sata ports on board, which I was told in 9.3 days to only use 8 of the ports due to the 4 port marvel controller causing performance issues. I have added a adaptec 24 port raid controller. I have 28 drive online of which 8 are currently used. I have the data backed up. I'm ready to start fresh. 1) should I stick with 9.3 or go to what version. 2) Should I consider getting 2 more drive to make it 32 (without using the marvel 4 port controller) or is the marvel controller problems fix in a newer release and just get 6 more drives to max out the ports? 3) with all these drives should I consider an l2arc? 4) what raid level Raidz3? It's primary use is plex and mythtv. Any advice would be great.

 

[danb35]

I have added a adaptec 24 port raid controller.

This is almost certainly a bad idea.

As to pool layout, you could do five, six-disk RAIDZ2 vdevs, or add two disks and do four, eight-disk RAIDZ2 vdevs. You could probably even go with three, ten-disk RAIDZ2 vdevs. Smaller vdevs keep your data a little safer, and more vdevs give you more IOPS, but those tend to lose more space to redundancy compared to fewer, larger vdevs.

with all these drives should I consider an l2arc?

Unlikely you have enough RAM to make a L2ARC a good idea.

 

[Allan_Guy]

I have 32 gig of ram but can consider 64 in the future. Right now it is a Raidz1 so 8 disks with 1 redundant. I had spare disks hanging around so I wasn't so concerned with raidz1 then. But, now with all the disks I have, I just want to maximize space and good performance. But, at least raidz2 or 3?

 

[danb35]

From what I've seen around here, 64 GB is the minimum RAM before you might consider an L2ARC device. Even at that point, keep an eye on your ARC stats. If your hit rate is already 90+%, you don't stand to gain much from an L2ARC.

 

[Allan_Guy]

OK, I'll check hit rates after setup. I also boot from a internal thumb drive. Should I consider a small SSD on the Marvell SE9230 for a boot controller? Every time I loose power I have to press f11 to get it to boot from the USB. Any word on the Marvell SE9230 working well in newer versions? I also have a generic question; if I move a drive from an existing array to another raid controller, Will freenas identify it and work with it? Or will it cause a problem?

FYI, Thanks Cyberjock for that PDF explaining vdevs and zpool, etc... Very helpful.

 

[depasseg]

Double check the Adaptec card is configured as an HBA. (no RAID, just a controller). If it's not, then get a proper HBA. You will likely run into issues if you incorporate any sort of HW RAID. Especially this question:

I also have a generic question; if I move a drive from an existing array to another raid controller, Will freenas identify it and work with it?

Yes, assuming that FreeNAS has direct access to the drives and there wasn't a HW RAID layered underneath ZFS. Of course, for the data to be in tact you need to move all the drives from the pool (not just one).