Adding a drive to a RAIDZ1 pool

[pico1180]

I know this question has been asked 27 different ways, but I was hopping for some clarification.

I just want to be able to expand my current RAIDZ1 pool.

The literature I have come across so that is a no-go. I need to create a new pool and add it to the existing pool. The closest acceptable solution I have found is to make a mirror and then add that pool to the existing pool.

My concern is, when I try to add a new vdev, Truenas yells at me saying my new vdev has to have the same amount of drives as my current vdev. That is just weird caveat to throw on something. It looks as though Truenas will allow me to add a vdev consisting of a single drive, but it makes me sign a waiver before doing it. Of course it has me scared to do it so I wont.

I literally get three different warnings. One says adding vdevs with different numbers of disk is not recommended. One warnings says, this vdev requires at least 3 disks. And the third warning says, you can do this, but you have to sign your life away.

So, what's the deal? Why can't I just add a another drive like a normal RAID 5 would let me? What happens if I click throw all the warnings and add a single drive anyways? Will it let me and what is the consequence?

 

[Ericloewe]

You typically do not want grossly mismatched vdevs, for a variety of reasons, even though they're technically supported.
In particular, if you add a single disk as a new vdev, that new disk must not fail, because it will contain data that exists nowhere else on the pool.

 

[ChrisRJ]

So, what's the deal? Why can't I just add a another drive like a normal RAID 5 would let me?

Because for the intended use-case of ZFS that was not relevant. ZFS was developed in the early 2000s for large (like at least hundreds of drives) storage systems for enterprises. In such a scenario you do not a single drive. Even adding "only" 8 drives was/is not that common. You rather add an entire JBOD enclosure with 40+ drives.

In contrast, a RAID controller is for a single system and a relatively small number of drives. Besides, with current HDD sizes you don't want RAIDZ1 (or RAID-5 for that matter) but RAIDZ2 (RAID-6 respectively) or higher.

What happens if I click throw all the warnings and add a single drive anyways? Will it let me and what is the consequence?

You DON'T want to do that !!!!!!!!

You would extend the pool with an additional vdev. But that vdev would be only the new single drive. So it has zero redundancy. If that new drive fails, your pool is gone. And new drives fail with a much higher probability than those that have been running a year or longer.

 

[artlessknave]

Why can't I just add a another drive like a normal RAID 5 would let me?

because RAIDz is not RAID. it's a completely different technology that implements a function that is similar at a high level.

if you remove a disk from a RAID, and then readd it, it will resync the entire disk.
if you remove a disk from a zfs pool and readd it later, zfs will scan it and then update it as needed.
very different

RAID copies everything. data, empty space, bad data. everything. it's independent of the filesytem and typically has no idea whats on the filesytem.
zfs is an integrated storage manager it knows what it's stored, checksums all of it, and only resilvers data as needed ; RAIDz (and mirrors) merge the filesytem with the volume managment with the disk management. LVM, for example, does this with 4 whole separate tool chains (pvs, vgs, lvs, mkfs)

zfs was designed for the enterprise space; it was designed for and assumes: you have backups, can just throw hardware at it, and plan years ahead with appropriate enterprise budgets. there has been recent effort to add raidz expansion but that is being written entirely as open source, and will likely be awhile still. zfs was designed by an enterprise before oracle closed the source.

it's like driving an manual transmission (RAID) vs automatic transmission (LVM) vs a constantly variable transmission (ZFS); they all make the car go, but HOW they do so is very different.
RAID only handles one part, you must do the rest yourself.
LVM handles all parts but in separate stages.
ZFS does everything, usually in one command.

 

[Pedulla5]

You typically do not want grossly mismatched vdevs, for a variety of reasons, even though they're technically supposed.
In particular, if you add a single disk as a new vdev, that new disk must not fail, because it will contain data that exists nowhere else on the pool.

What ever happened with this project?

OpenZFS RAID-Z Online Expansion Project Announcement – FreeBSD Foundation

www.freebsdfoundation.org

 

[Ericloewe]

It's almost there, after many twists, turns and contributions.

 

[beagle]

You can see the latest update at:

OpenZFS Developer Summit 2023 Talks​

OpenZFS Developer Summit 2023 Talks - OpenZFS

openzfs.org

OpenZFS Developer Summit 2023

schedule here: https://openzfs.org/wiki/OpenZFS_Developer_Summit_2023

www.youtube.com

 

[asap2go]

It's almost there, after many twists, turns and contributions.

And by almost there we mean it will be ready end of 2024.
Probably.
Like quite likely.
If testing goes as intended.
Definitely 2024.
It's really happening now.

 

[Ericloewe]

Although a certain amount of skepticism is a good thing, as the presentation indicates, the work is very close to complete. The PR has a few relatively minor issues that have been pointed out, but nothing stood out to me as problematic.
So, I would be genuinely surprised if this is not integrated before the end of the year.

 

[Arwen]

Although a certain amount of skepticism is a good thing, as the presentation indicates, the work is very close to complete. The PR has a few relatively minor issues that have been pointed out, but nothing stood out to me as problematic.
So, I would be genuinely surprised if this is not integrated before the end of the year.

And that means conservative people can start using it at the end of 2024 . After the .1 and .2 revisions of the feature come out fixing misc. issues, bugs, and heaven forbid, data loss.

 

[Ericloewe]

That, of course, is fair and a separate whole part of the lifecycle of any major feature.

 

[Whattteva]

I'm probably gonna let it cook in the wild for at least a year first and let everyone else work out the bugs. Not in a rush.