How does mirror work with FreeNAS and ZFS?

[rangopango]

If I have four 8TB drives, and I set up two groups with two drives each, I would have to lose two drives in one group to lose all of my data right? Or do I only lose data in that particular group while the other group remains?

 

[Redcoat]

edit: ops, I wrote FreeNASA in the title, I meant FreeNAS

You can edit it - see "Thread Tools" at upper right side.

 

[scrappy]

If I have four 8TB drives, and I set up two groups with two drives each, I would have to lose two drives in one group to lose all of my data right? Or do I only lose data in that particular group while the other group remains?

You could lose a drive from each mirrored pair of drives and still have a functioning zpool. But if you lost two drives from the same mirrored pair then you are out of luck.

 

[rangopango]

That sucks... two drives in each group out of two groups offers a little bit more storage, faster performance and faster resilvering (in case a drive fails) compared to Raidz2, but the latter offers more protection because losing the two wrong drives in a mirror will be game over.

This means I'll have to back up my mirrored setup more often and pray that I won't lose the wrong drives at once :D

Another question: if I lose one drive, and it's being rebuilt after I add a new drive, what will happen if there's error in the data? How would it be corrected?

 

[pschatz100]

With four drives, your configuration options are as follows:
1) Two mirrors, one mirror each into a pool. Result: two pools.
2) Two mirrors, stripe both mirrors onto one pool. Result: one pool.
3) One RaidZ2 with four disks. Result: one pool.

With the first option, you can lose one disk from a mirror without losing data. If you lose two disks from the same mirror, then that pool is gone. The other pool is unaffected.

With the second option, you can lose one disk from each mirror without losing data. If you lose two disks from the same mirror, then the entire pool is lost.

With the third option, you can lose any two disks without losing data.

What is best? Well, it depends on your priorities. The write performance for a mirrored pair will be better than RaidZ2. Option 2, above, will have better write performance than Option 3, but Option 3 will be somewhat "safer" in that any two disks can fail before data is at risk.

For a media server or backup system where write performance doesn't need to be optimized, I would consider RaidZ2 to be the better option. For a system hosting VM's, or for which write performance is critical, then mirrors might be a better choice. For a system that does both, one could set up a RaidZ2 pool for data, and a mirror for the VM's.

 

[rangopango]

I'm partial towards 1) for a number of reasons.
Scrubbing should be faster, resilvering should be faster, less strain on the drives during rebuild, and my entire pool won't disappear if the wrong two drives fail. The only problem is that it'll be two pools which makes it harder to decide where to put data etc. How much of a problem will fragmentation in ZFS become after using the same pool for one year or more? I heard the best solution is to create a new pool and move over the data to it

 

[pschatz100]

There is no simple perfect answer. The more pools you have, more issues with management, fragmentation, etc. Personally, I like to keep things simple. RaidZ2 for data. Mirror for VM's.

... and good backups.

 

[rangopango]

How long does it take to rebuild a 10TB pool if one drive fails out of five drives in Raidz2 with regular 7200RPM drives?

 

[Bidule0hm]

I have a 37 % used 13.1 TB pool (so 4.7 TB of data) with 8x 3 TB drives in a RAID-Z3 vdev and I'm currently resilvering a drive since 40 min and there's 2 hours remaining.

So with a smaller RAID-Z2 pool and faster drives (mine are NAS drives so 5400 RPM) it should take even less time than that.

Keep in mind ZFS only resilvers the used space so the fuller the pool the slower the resilver.