Which RAID level should I use for an 8-bay case?

[PerpetuallyConfused]

Hello, I'm new to the world of FreeNAS and building a NAS in general. Actually first and foremost I will not be using FreeNAS and ZFS on my first build because it's quite intimidating in terms of amount of knowledge uptake required and I will be using Linux and standard RAID with mdadm (which I am not an expert with in the first but until I make the jump in the future to FreeNAS it's all I'm comfortable with atm) so if my question is not allowed due to being off-topic just let me know and I will only ask FreeNAS related questions here...

I am planning on purchasing an 8-bay case such as the Silverstone CS381 or the U-NAS NSC810A. First off I don't see any reason not to fill up all the bays, so I will be using as many of the 8 drive slots as possible. I found some "triangles" while reading about RAID telling you to pick 2 of the 3: Fast, Cheap, Fault tolerant, and my choice is Fault tolerant first, then cheap, with speed being the least important.

I've narrowed down the possible RAID levels to only three: 6, 10, and 60.

Here's what I've picked up so far:

RAID 10: Fastest of the three, no parity calculations and quick rebuilding time, at best can tolerate 1 failure in each mirror, with a maximum 50% (four) disks failing as long as each mirror still has a remaining disk, otherwise 2 disks in the same mirror will bring it down, quite inefficient in terms of space - 50%, n/2 (but that's OK)

RAID6: Slower due to parity calculations, but RAID5 is no longer an option with large disks and chances of failure during rebuilding. Can tolerate any 2 disks failing. Minimum 4 disks. Actually a bit confused right now because I didn't know RAID 6 could scale upwards but apparently the limit is quite high (16, or 32, depending on sources I've found). It seems (uneducated guess) to me that a 4 disk RAID6 would be more safer than a, say, 8 disk RAID6, because a higher percentage of the space is used for parity, but I'm not sure. Anyhow obviously space utilization would be higher at n-2, so with 8 drives at max 75% usable space.

RAID 60: Basically a RAID0 of two RAID6's. I've read some sources saying it's better than RAID6 in every way (including rebuilding), and it's somewhere between RAID6 and RAID10. It can survive 2 failures in each RAID6 leg, so a total of any 4 disk failures in my case. So in some ways similar to RAID10 except RAID10 can only tolerate 4 specific disk failures, or at worse 2 will kill the array. Again quite inefficient at 50% space utilization like RAID10 at n/2

Also I suppose I could use less than 8 slots and make a smaller array but use some of the slots for a hot spare if necessary (please advise)

The biggest issue would be low levels of usable space as low as 50% for RAIDs 10 and 60 and since the cases only have 8 slots, upgrading would mean enlarging each drive in the array rather than adding more drives. But that is not too big of a problem with 10TB + drives being relatively affordable and hdd capacity growth in the future.

So my question is, for an 8 drive case, should I use RAID 6, RAID 10, or RAID60, and should hot spares be involved? I am leaning towards RAID 60.

 

[Heracles]

Hi,

First thing is you should never do any Raid when using FreeNAS. Raid is performed by hardware Raid controllers and they put hard drives out of reach of FreeNAS and ZFS. ZFS must have direct hardware control of its drives.

In ZFS, there is no Raid6. The structure offering 2 redundant drive per group is Raid-Z2.
In the same way, there is no Raid60. The equivalent is a pool containing 2 (or more) Raid-Z2 vdevs.
Same thing for Raid1 and Raid 10 : In ZFS, they are mirrors and a pool of 2 (or more) mirrors.

As for the recommended setup for you, it depends of your needs. Here, my 8 bay server is configured with mirrors because I do iSCSI and offer storage for my ESXi server. My data are replicated according to the 3 copies rule (see my signature) and so the normal redundancy offered by mirror is enough. I also keep a cold spare but no hot spare.

Should you do only low level of IOPS, a single 8 drives Raid-Z2 is your second option. 2 Raid-Z2 vdev would be non-sense. Same space as mirrors but only half the vdev, so basically half the speed.

 

[HoneyBadger]

Not sure how FreeNAS would be considered more confusing than a Linux mdraid setup - there's some peculiarities to ZFS itself as a copy-on-write filesystem, but if this is just for storing general media (most home use cases) most of them can be mitigated with "don't go above 80% used space."

In your case, only 8 drives leads to a RAID6 if you're after capacity first. RAID60 would come into play around 12 drives in my opinion (two 6-drive R6's)

But there's no way to "migrate" from mdraid to ZFS later. Is there something in particular that's challenging you with the FreeNAS/ZFS understanding? Even if you end up sticking with the mdraid solution in the end it's valuable feedback.

 

[blanchet]

For a home user, a stripe of mirror may be the less expensive option because you can buy only 2 new hard disk when you need more space.

• if you have free slot, you can just buy two new disks
• if you have no free slot, you can replace 2 smaller disk by 2 bigger ones

On the other hand, for a 8-way Raid-Z2 you have to buy the 8 disks at the first time, even if you do not need immediately the full disk space.

In my opinion, you need Hot Spare only when you cannot go quickly to the server when a disk is broken.
Cold Spares require neither electricity nor a free slot, and they do not wear out.

 

[garm]

I wouldn’t fill all bays, I would put 6 drives in a RAIDZ1 vdev (based on your requirements) and leave two bays for resilvering tasks. I would also put a few SSDs velcro’ed around the chassi for a dedicated jail pool or fusion pool. To achieve this I would use a LSI 9211 and a motherboard with the appropriate amount of SATA ports.