SOLVED Does FreeNAS support some equivalent to RAID 1E?

[MenhirMike]

Right now, I'm running a Windows-based system with an LSI 9211-8i in IR Mode, with 3x 4 TB Drives in a RAID 1E. I can't put in more drives right now (for a RAID 10), and I definitely don't want to do a RAID 5 on such a big array.

I'm considering switching to FreeNAS (and flashing the controller into IT mode), but I'm not sure if there's an alternative to Raid 1E.
Everything I saw that talks about RAIDZ1 basically makes it look like RAID5 (so 2/3rds of the capacity, but with super expensive rebuilds) instead of a Raid 1E (1/2 of the capacity, cheaper rebuilds)

 

[danb35]

How is RAID 1E different from RAID 1? Because ZFS (and thus FreeNAS) supports mirrors, which are comparable to RAID 1. We don't recommend RAIDZ1 on drives larger than about 1 TB, but I wouldn't call the rebuilds "super expensive".

 

[MenhirMike]

RAID 1 uses exactly 2 drives, while 1E uses any odd number of drives, so 3, 5, 7, etc. and it allows any number of Non-Adjacent drives to fail.

Like RAID 1, RAID 1E leaves 50% of the capacity, compared to 66% for RAID 5 (so 12 TB Net turn into 6 TB rather than 8 TB)

Basically on a 3 Disk RAID, the data is laid out like this:

Code:

		 | Disk 1 | Disk 2 | Disk 3 |
---------+--------+--------+--------+
Stripe 1 |   1	|	1M  |	2   |
Stripe 2 |   2M   |	3   |	3M  |
Stripe 3 |   4	|	4M  |	5   |
Stripe 4 |   5M   |	6   |	6M  |
---------+--------+--------+--------+

If Disk 2 fails, the data can just be rebuilt from the mirrors on Disk 1 and 3.
If this were a 5 disk RAID 1E, then disks 2 and 4 could fail - basically, any number of non-adjacent disks can fail since the next disk over will have either a mirror or the original data.

The reason I call RAID 5 rebuilds expensive are because on RAID 5 with a failed Disk, it's required to read the full size of each disk. So with 4 TB disks, I have to read 4 TB on Disk 1 and 4 TB on Disk 3. The Non-recoverable read errors per bits read is usually 10e-14, so the chance is pretty high that one of the reads will fail. With a RAID 1E, I only need to read half the capacity on Disk 1 and 3. (Of course with big enough disks, it's still a risk. But RAID is not backup anyway).

(Note: I am not sure how RAIDZ1 rebuilds work. The articles that I found describe it as "like RAID5", but I don't know if it just looks like RAID5, or if it uses the same parity scheme that requires reading all the values in a stripe to reconstruct the missing disks data)

For reference, a 5 Disk RAID 1E:

Code:

		 | Disk 1 | Disk 2 | Disk 3 | Disk 4 | Disk 5 |
---------+--------+--------+--------+--------+--------+
Stripe 1 |   1	|	1M  |	2   |   2M   |	3   |
Stripe 2 |   3M   |	4   |	4M  |   5	|	5M  |
Stripe 3 |   6	|	6M  |	7   |   7M   |	8   |
Stripe 4 |   8M   |	9   |	9M  |   10   |	10M |
---------+--------+--------+--------+--------+--------+

 

[Stux]

Well, its not out of the box, but if you wanted to, you could structure an array like that... basically partition your drives, then assemble mirrors.

ZFS will distribute writes across all the mirrors which are added to a pool, basically sending new writes to the first mirror which returns...

Or you could just use RaidZ1/Z2 ;)

ZFS only reads *used* data to reconstruct

 

[MenhirMike]

ZFS only reads *used* data to reconstruct

Thanks, that's a pretty significant piece of information, since that makes it really good. Hardware RAID doesn't know about the file system, so it always has to rebuild the whole disk. But if it only reads the used data, then RAIDZ1 would be the clear winner here.

 

[danb35]

The reason I call RAID 5 rebuilds expensive are because on RAID 5 with a failed Disk, it's required to read the full size of each disk.

This isn't the case with any ZFS RAID arrangement--rebuilds read only the used blocks. Since ZFS is both the filesystem and the volume manager, it knows which blocks are used and which aren't.

 

[Arwen]

Going back to RAID-1E, no ZFS does not support that concept natively. (See @Stux's comment for non-native... though rebuilds with that strange setup may be painful...)

ZFS Mirrors are at least 2 disks.
You can have 3 disks in a Mirror if your use case is heavy on the reads. Or you need higher levels of redundancy.

ZFS will stripe across vDevs, (virtual devices), if you have more than one. For example, 2 x 2 disk Mirrors.

 

[MenhirMike]

Cool, I think I might just do a RAIDZ1 then. I want fault tolerance for 1 drive, but more than 4 TB of space.

Will play around in a VM with a 3 Disk RAIDZ1 next.

 

[Arwen]

Cool, I think I might just do a RAIDZ1 then. I want fault tolerance for 1 drive, but more than 4 TB of space.

Will play around in a VM with a 3 Disk RAIDZ1 next.

Yes, play around with ZFS. It's different enough that some thought is required to get what you want out of it.