Expand ZFS Mirror

[jiblett1000]

Hello all,

I currently have two 1TB hard drives in a ZFS mirror setup on my nas box. However, I have run out of space and need to expand. I was thinking of buying two 3TB drives (6TB more). My question is, can I just add the two new 3TB drives to the existing mirror or do all my drives have to be the same size?

-Thanks

 

[JaimieV]

Neither! Go read Noobsauce80's presentation at the top of the FreeNAS4Noobs forum, then come back and ask any further questions.

 

[jiblett1000]

Ok, read it. So, adding drives to an existing Vdev isn't possible as it said in the slideshow so it's not t all possible to expand a ZFS Mirror. However since Vdev's make up zpools.....Given the situation below.

2 x 1TB drives (mirrored, 1TB usable space)

2 x 3TB drives (mirrored, 3TB usable space)

If there are two different ZFS Mirrors, can I combine them into one zpool? so the data will still be mirrored but split across vdevs?

 

[JaimieV]

:) You can indeed - in that case you'll see one volume of 4Tb, with a slightly more complicated than simple mirror resilience (allowing failure of one or a good choice of two drives).

Alternatively you can form the new mirror as a separate zpool if you'd rather. Copy the data over, then remove and reassign the old 1Tb drives as being a backup store for the new system. That's my normal upgrade path.

 

[jiblett1000]

Ok, think I got it now. So, ideally the best thing to do would be to get 4 x 3TB drives, take the data off and then create a mirror with all four. That would allow failure of any two drives instead of select ones.

If I did the second route that you mentioned I would only get 3TB from that configuration through right?

 

[JaimieV]

Second route: Yep, 3Tb from 2x3Tb in mirror.

If you get four drives, you're better using RAIDZ2 instead of mirrored pairs: in RAIDZ2 any two drives can fail simultaneously without loss of data, and you'll get the same 6Tb usable space (well, 5.4Tb - you know how marketing terabytes are shorter than real terabytes...).
You could also do just three 3Tb drives in a RAIDZ (==RAIDZ1), which would allow any one drive to fail without losing data, same as a mirror. Same capacity still.

The upgrade procedure would be to add the new drives, create the new zpool over them, copy the data, then decommission the old mirror.

And then have a ponder about how you'll backup the new enormous datastore you've just created.

 

[bollar]

The one advantage of striped mirrors is that they're potentially much faster than raidzx arrays. If speed is an issue for you, that's the reason to keep then in consideration.

 

[JaimieV]

Depends how low-end your kit is. Mine's pretty low, I'm using a couple of HP N36Ls with dualcore 1.3GHz, and RAIDZ2 is no different than striped mirrors - from the point of view of the network client workstation. The GigE is the rate limiting step.

I didn't think to test local read/write speeds, since for my domestic use it holds very little importance.

 

[Stephens]

The one advantage of striped mirrors is that they're potentially much faster than raidzx arrays. If speed is an issue for you, that's the reason to keep then in consideration.

Is there a recommended RAID-Z1 or RAID-Z2 configuration that'd serve up less than gigabit speeds? My 6-drive RAID-Z2 is way faster than my gigabit connection. The only time the excess speed seems to come into play is during scrub/resilver. That also happens to be the only time I kind of wish I had a faster CPU.

 

[bollar]

Is there a recommended RAID-Z1 or RAID-Z2 configuration that'd serve up less than gigabit speeds? My 6-drive RAID-Z2 is way faster than my gigabit connection. The only time the excess speed seems to come into play is during scrub/resilver. That also happens to be the only time I kind of wish I had a faster CPU.

Yes. I think so -- iSCSI and NFS loads come to mind as potential issues. Of course, the other advantage of mirroring is that the array can be expanded horizontally or vertically in groups of two.

But everything is a tradeoff, isn't it? Mirroring loses half the array to redundancy, which is a killer at larger array sizes. It also has a higher MTTDL than RAIDZ2 at the four drive configuration we're discussing.

 

[jiblett1000]

I wish I could find this one post again. Someone had done some benchmarks on different configurations. Mirroring was by far the fastest which is why I'm inclined to use that (along with the safety advantages). Although, I feel that when I transfer things via NFS it feels like it is probably the bottleneck in my setup. Is there anything faster than NFS?

 

[bollar]

There are several threads on that subject, as well as blogs from Sun/Oracle gurus on the topic, but here's the one I started: http://forums.freenas.org/showthread.php?9989-Creating-a-Set-of-Benchmarks

Without tweaking, NFS & ZFS seems like a very slow configuration. CIFS/SMB are single threaded and bound by CPU speed. And everyone says AFP is "slow". I can tell you that I can get 100MB/sec with SMB and AFP after throwing enough CPU and mirrored/striped drives at the problem -- NFS, I haven't cracked (but to be fair, I don't use NFS in production, so haven't put as much work into the problem as I could).

 

[campbela]

:) Alternatively you can form the new mirror as a separate zpool if you'd rather. Copy the data over, then remove and reassign the old 1Tb drives as being a backup store for the new system. That's my normal upgrade path.

Could you also replace each of the 1TB drives with a 3TB drive, resilvering in-between, which would keep everything as it is (except for more disk space :)). I guess the downside to this would be the risk of the other 1TB drive failing while resilvering.

 

[JohnnyFreeNAS]

in the guide i read this..

Code:

to extend a ZFS mirror, add the same number of drives. The resulting striped mirror is a RAID 10. For example, if you have 10 drives, you could start by creating a mirror of two drives, extending this mirror by creating another mirror of two drives, and repeating three more times until all 10 drives have been added.

after reading this thread I'm a little confused. i have 2 drives in a mirror i will be adding 2 more equal size drives to the pool. am i to assume by the guid that i would end up with double the capacity in a raid 10 state? (mirrored&striped)?

 

[Robert Trevellyan]

Yes.

 

[Jailer]

Holy necro post!

 

[danb35]

Holy necro post!

Seriously. Seems like a lot of that going around recently.