RAIDZ expansion, it's happening ... someday!

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,457
I don't understand this going over 12 talk.
Then you've got a lot of reading to do. Start here:
https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

In short: you should not have more than 12 (really, more than 10) disks per vdev.

Edit:
I am planning on a complete new build from the ground up once this gets implemented.
You should be aware (in case you aren't already) that it's entirely possible to expand a pool already, and it has been for many years. In the scenario you're suggesting, where you're planning to double your pool capacity, this is what you'd want to do anyway--simply add another vdev to your pool. FreeNAS has had this capability through the web GUI for a long time. What isn't currently possible (but is coming, and is the subject of this thread) is to expand a RAIDZn vdev--to turn a six-disk RAIDZ2, for example, into a seven-disk RAIDZ2.
 
Last edited:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Man, even 12 disks in a RAIDZ2 with a single underlaying VDev would have me a bit worried. Is that common practice?

I have 4x 4 TB now in a RAIDZ1 for media. Let's say I would like to expand. What would be the better option:
  1. Create a new VDev and add it to the pool
  2. Move the data and create a new RAIDZ2 pool and import again
Not to prolong this tangent too much, so that we can keep this thread on topic, but "better" depends on your requirements. 1. is more straightforward, but 2. means RAIDZ2, which is more reliable.
 

Yorick

Wizard
Joined
Nov 4, 2018
Messages
1,912

Z300M

Guru
Joined
Sep 9, 2011
Messages
882
You can keep tabs on it here: https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/mobilebasic and on the spreadsheet: https://docs.google.com/spreadsheets/d/1CFapSYxA5QRFYy5k6ge3FutU7zbAWbaeGN2nKVXgxCI/htmlview

It’s being developed, no PR (pull request) yet. I wouldn’t expect a huge amount of movement until after ZoF is stable.
I currently have two pools:

Pool 1: 6 x 6TB drives in RaidZ2. From reading through this thread, it seems that adding a drive to give 7 x 6TB drives in RaidZ2 would be straightforward (once the feature is available).

Pool 2: Two vdevs, each of 7 x 2TB drives in RaidZ2. Once this new feature is available, could I replace all the 2TB drives in one of the vdevs by larger drives (one at a time, as at present), then detach the other vdev altogether (maybe would have to be one drive at a time?)?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Pool 2: Two vdevs, each of 7 x 2TB drives in RaidZ2. Once this new feature is available, could I replace all the 2TB drives in one of the vdevs by larger drives (one at a time, as at present), then detach the other vdev altogether (maybe would have to be one drive at a time?)?
Nope. Different feature, device removal, but it only works for pools made up entirely of mirrors. Any RAIDZ is disqualifying for now.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,457

Yorick

Wizard
Joined
Nov 4, 2018
Messages
1,912
Matt Ahrens is back to working on this feature. There are a couple minutes on it during the last meeting. Alpha code now or within the week; don’t use on anything you care about, you will have to destroy your pool later bcs of ondisk format issues. That said, he encourages testing this with diverse workloads. Comes to ZoL for testing.
 

Arwen

MVP
Joined
May 17, 2014
Messages
3,600
...
it only works for pools made up entirely of mirrors.
...
...or single disks.
A single disk is just a degenerate case of mirror vdev - a one-disk mirror. :)
@jgreco, that's degraded mirror! All striped disks are degraded until you add the appropriate amount of mirrors.
We just don't get warning or alarms on striped disks, awaiting their mirror(s).

As for the RAID-Zx expansion, seems a little late. Kinda expected the preview 6 months ago.
Based on what we know now, I would not expect it to be user ready for at least 6 more months.
And for production, (meaning people who care more about uptime than features), a year.
 
Last edited:

Yorick

Wizard
Joined
Nov 4, 2018
Messages
1,912
Well, raidz expansion is nice - and, it seems to me more of a FreeNAS feature than an enterprise ZFS one. DRAID is likely much more interesting to enterprise, and even that is taking its time.
I’m perfectly fine with raidz expansion taking another year. When I built my home system, I figured two to three years before I would want to or need to add storage. I’m right on track so far. I did bet on raidz expansion to be available by then, which, well, that may still come to pass.
 

anmnz

Patron
Joined
Feb 17, 2018
Messages
286
A single disk is just a degenerate case of mirror vdev - a one-disk mirror. :)
If more people understood this, there'd be a lot less confused talk about "stripe" vdevs, or "striped pools" as if they were a different kind of thing from any other pool.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
If more people understood this, there'd be a lot less confused talk about "stripe" vdevs, or "striped pools" as if they were a different kind of thing from any other pool.

Well I'd say that's actually the flip side of the coin I was discussing, but it is definitely right up there in the things that people don't grasp about pools. :smile:
 

ethanf

Dabbler
Joined
Jul 1, 2017
Messages
26
I see recent comments on the PR but still feels like this is dead? I've been dreaming of the day to add an extra disk to my vdevs. It would definitely make it easier to expand incrementally versus buying 10 drives at once to add another full vdev.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
Sometimes complicated stuff takes a long while. I can't tell if it's dead or just taking a long while. I've got active projects that are older than this though, so maybe don't despair just yet.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
As I understand it, FreeBSD was going to be the reference implementation, but then ZoF happened and work was basically paused.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
No, but lost in the noise until the dust settles seems likely.
 
Top