Introduction to ZFS

Introduction to ZFS Rev 1d)

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Maybe also
8. Mentioning that there is no raid 01 analogue in ZFS world.

Sent from my phone
As in "mirroring vdevs instead of striping them"? I think that's fairly clear in the second paragraph of the "ZFS Pools" subsection.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
"okay, so how do I configure snapshots, they're awesome and I want them" and look at the manual and see that section so-and-so tells them what they need to know.
The problem with that is that the manual isn't that great as a "how-to" document. It's a very thorough reference (far better than most of what I see in the F/OSS world), but after installation, it doesn't do much to walk you through common tasks.
 

pro lamer

Guru
Joined
Feb 16, 2018
Messages
626
The problem with that is that the manual isn't that great as a "how-to" document. It's a very thorough reference (far better than most of what I see in the F/OSS world), but after installation, it doesn't do much to walk you through common tasks.
Anyway we have some resources that do. Thinking of Uncle Fester's guide. Maybe we need a newer version of it?
.
Sent from my phone
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456

pro lamer

Guru
Joined
Feb 16, 2018
Messages
626
We probably do. That's why it's a wiki. (-:
I guess this topic is worth its own thread "why don't we update Uncle Fester's guide often enough"... :/
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Ericloewe updated Introduction to ZFS with a new update entry:

Rev 1c)

This is Revision c) of the document. I somehow forgot to mention the FreeNAS User Guide in the Further Reading section - that has been fixed. I also added a section on the recordsize property.

Besides that, a footnote was added to the dedup subsection guiding people towards FMAZ if they really want to know more about dedup. I also added the links to the Resources section and to the Resource page to the first page of the document.

Read the rest of this update entry...
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175

BaoNT

Cadet
Joined
Dec 7, 2019
Messages
2
Hi,

I wonder if anyone knows about the status of this?
The most important limitation of RAIDZ is that vdevs are immutable. You cannot add or remove disks from a RAIDZ vdev, though work is ongoing to allow for expanding RAIDZ vdevs (but not removing disks). It is best to plan ahead with this limitation in mind.

Thanks,
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Status is unchanged.
 

NickF

Guru
Joined
Jun 12, 2014
Messages
760

Here is a variation of this I have written
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
Here is a variation of this I have written
Why would you include a picture of an SMR drive (WD EFAX) with a title: A WD Red Spinning Hard Drive?

That's something that some folks may see as an endorsement of that model to be used with ZFS, which hopefully we all know is a disastrous idea.
 

NickF

Guru
Joined
Jun 12, 2014
Messages
760
Why would you include a picture of an SMR drive (WD EFAX) with a title: A WD Red Spinning Hard Drive?

That's something that some folks may see as an endorsement of that model to be used with ZFS, which hopefully we all know is a disastrous idea.
Fixed :) Sorry for confusion, was a mistake in editing. Was placed there to link to the SMR/CMR article.
 

echadwick

Cadet
Joined
Dec 29, 2021
Messages
4
Loved this guide, especially this bit:
Disks are evil. They lie about their characteristics and layout, they hide errors, and they fail in unexpected ways. ZFS means no longer having to fear that your disks are secretly plotting against you. Yes, your disks are plotting against you, but ZFS exposes their treachery and puts a stop to it.


I'm trying to understand RAIDZ, and wondering if this guide could use further clarifications? Could certainly be I just need to read more, but the following bits are confusing at this point for me.

A pool contains one or more vdevs, short for virtual device, and is striped across them. That means that data does not overlap across them and the failure of one vdev implies data loss. Again, if one vdev is lost, the pool fails.
If my backup data was a single file, e.g. a JPG image file, would a vdev be analogous to a single file? Two disks, mirrored, would store two copies of the JPG? Is a vdev one complete copy of the data I wish to back up in a particular pool? Wondering if a diagram might help here?

Again, if one vdev is lost, the pool fails.
Does this mean if one drive in the array fails, the pool fails, and thus data is lost? Seems contrary to all the subsequent info about mirroring, RAIDZ, etc.

A RAIDZn vdev can tolerate up to n disk failures without data loss.
What are the minimum # of disks required for each RAIDZ? I'm guessing RAIDZ1 requires a minimum of 2 drives? RAIDZ2 min is 3? etc.?

RAIDZ does not have a write hole
What is a write hole?

Thanks for the great guide. Reading, and re-reading it, alongside the main iXsystems doc.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Loved this guide, especially this bit:
As I say, I can't claim authorship of that one. Be sure to check out the ZFS books.

A pool contains one or more vdevs, short for virtual device, and is striped across them. That means that data does not overlap across them and the failure of one vdev implies data loss. Again, if one vdev is lost, the pool fails.
If my backup data was a single file, e.g. a JPG image file, would a vdev be analogous to a single file? Two disks, mirrored, would store two copies of the JPG? Is a vdev one complete copy of the data I wish to back up in a particular pool? Wondering if a diagram might help here?
I see how the sentence is confusing. I'll rewrite it as:
A pool contains one or more vdevs, short for virtual device, and data is striped across these vdevs. That means that data is distributed across them and the failure of one vdev implies data loss. If one vdev is lost, the pool fails.

To answer your more immediate question, pools are made up of vdevs. Each vdev can be either a single disk, a mirror of disks, a RAIDZ vdev, or a dRAID vdev (this last one is a new option). If you lose any vdev (i.e. you lose the single disk, all disks in a mirror, or more than n disks in a RAIDZn vdev), you lose the pool. It's not 100% cut and dry, but it's a close enough approximation.

What are the minimum # of disks required for each RAIDZ? I'm guessing RAIDZ1 requires a minimum of 2 drives? RAIDZ2 min is 3? etc.?
Strictly speaking, yes, but those options are silly. In practice, n+2, otherwise you'd just use a mirror, which would be faster and simpler.

What is a write hole?
Traditional RAID controllers are vulnerable to ending up with an invalid state where part of the data is new data, which was being written, and the rest is old data. This can be caused by something like a power failure.
 

echadwick

Cadet
Joined
Dec 29, 2021
Messages
4
Strictly speaking, yes, but those options are silly. In practice, n+2, otherwise you'd just use a mirror, which would be faster and simpler.
If I understand correctly there are different benefits between the two?

Mirror with 3 drives offers faster performance and more drives can be added at any time, but RAIDZ2 with 3 drives offers more capacity since it's striped?

Do they both offer the same fault tolerance? So is it really just a choice of perf/mutability vs. capacity?
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
Mirror with 3 drives offers faster performance and more drives can be added at any time, but RAIDZ2 with 3 drives offers more capacity since it's striped?
A three-disk mirror and a three-disk RAIDZ2 would have the same capacity, though the TrueNAS GUI won't let you create a three-disk RAIDZ2--both would have the capacity of the smallest disk of the three. Both would tolerate the loss of two disks, but you could turn the three-disk mirror into a two-disk (or four-disk) mirror at will if desired, while you couldn't do that with RAIDZ2.
 

echadwick

Cadet
Joined
Dec 29, 2021
Messages
4
Thanks. I thought RAIDZ was a striped array, so that means larger capacity than a mirrored array?

Feel free to ignore me. I think I just need to keep reading docs; these questions are likely answered already! Maybe my Qs are useful for adjusting a newbie-oriented doc.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
In a mirror, all components of the mirror have all data.

RAIDZ is analogous to RAID5/6, parity is calculated and together with the data gets written to "all" the disks (asterisks apply). Take a 6-wide RAIDZ2 vdev: Your data gets chopped up into 4 pieces and two more chunks of parity are calculated. These are written to the six disks and any two disks can fail.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,456
I thought RAIDZ was a striped array
RAIDZn is a striped array with parity, specifically with n disks' worth of parity. A RAIDZ2 array will have parity to fill two disks, so if there are only three disks in the array, there's only the space of one disk left for data.
 
Joined
Oct 22, 2019
Messages
3,580
@echadwick, in cases like this, it's sometimes more useful to consider visual-imagery of the analogy (like a conceptual understanding before diving into the more technical and nitty-gritty side of things.)

Think of a pool as a jigsaw puzzle, but if there is even a single piece of the puzzle missing, then you must throw the entire puzzle into the trash! No exceptions! It's all or nothing. Either it's a complete puzzle that is 100% useful, or if it's anything less than that, in which it's garbage and 0% useful.

What if the puzzle is just comprised of one single piece? Simple. No complexity. Problem is, if you lose just this one piece, it's game over! So you decide to make a copy of this single piece. If you lose one of these copies (of the single piece), your puzzle is still 100% useful.

A friend stops by and convinces you that your puzzle could use some improvements. The picture needs to be bigger and expanded! Right now it's too boring. So she convinces you to "improve" your single-jigsaw-piece puzzle into a double-jigsaw-piece puzzle! Now your puzzle is a complete 100% useful picture when two pieces are clicked together! (Plus it's a larger image too, way more interesting and fun!) And just like with the first piece, you take no chances and make a duplicate of this second piece, in case you lose one.

More and more friends come over, and they convince you to make your image larger and larger, which means your puzzle is comprised of more and more pieces.

The benefit? When your puzzle is 100% useful and "complete", it reveals the most beautiful, large image. Something to brag about and show off!

The cost? It's made of more pieces, which means you need to be more diligent and make sure you never lose any of these pieces. Sure, you've made duplicate copies of each of these pieces, which yields some safety, but the risk always remains.

Let's say the puzzle is so huge that it's made of 8 pieces total. All of these pieces you made copies of. If you ever lose any one of these eight pieces ("lose" as in you lose all copies of any particular piece), you must throw the puzzle into the garbage and forever lose the beautiful photo that it yields.

Think of these puzzle pieces as vdevs. Such "puzzle pieces" can be "mirrored" or "parity-based", depending on your preference of redundancy / performance / financial costs. :wink:

(Technically, I'm sure it's possible to recover some data with low-level forensics, but that's besides the point and should never be on the table as a consideration.)
 
Last edited:
Top