RAIDZ Volume Extension Cons

[fungus1487]

Hi All,

I have had a FreeNAS setup for 12-18 months now which has been great. I have a 5 disc RAIDZ volume which is now approaching 98% usage. I have another 5 drives to expand with, effectively doubling the amount of space I have, all my hardware is fine for this setup as I planned well ahead.

My question is what are the cons of expanding my current 5 disc RAIDZ into 2 x 5 disc RAIDZ configuration.

Currently this will transparently double my storage without having to setup other boxes to include another network folder. Is the risk of data corruption across the two RAIDZ configurations more likely to fail as a result of this? Should I set these 5 discs up as a seperate volume and not extend the first or are the risks negligible? Thanks for any help.

 

[Z300M]

Hi All,

I have had a FreeNAS setup for 12-18 months now which has been great. I have a 5 disc RAIDZ volume which is now approaching 98% usage. I have another 5 drives to expand with, effectively doubling the amount of space I have, all my hardware is fine for this setup as I planned well ahead.

My question is what are the cons of expanding my current 5 disc RAIDZ into 2 x 5 disc RAIDZ configuration.

Currently this will transparently double my storage without having to setup other boxes to include another network folder. Is the risk of data corruption across the two RAIDZ configurations more likely to fail as a result of this? Should I set these 5 discs up as a seperate volume and not extend the first or are the risks negligible? Thanks for any help.

I'm surprised nobody has replied already. I'm no expert, but I'm waiting for someone to tell you that you are already asking for trouble on two counts: (1) Letting your pool get to 98% full and (2) using only RaidZ rather than RaidZ2 or RaidZ3.

Simply adding another RaidZ pool isn't going to solve the 2nd problem at all and will solve the first problem only if, after you've added the new set of drives, you can find some way of redistributing the data, moving some files from the current 5-drive set to the new one -- and *I* have no idea how or even whether that can be done.

If I'm talking nonsense, somebody please tell me.

 

[joeschmuck]

Nope, you're not talking nonsense.

Also at 98% your ZFS pool is operating very poorly and you need to retain more than 10% free space at all times or ZFS will run in a particular mode where it's super slow. I don't recall the specifics but it's actually a software safety feature.

In my opinion the best way to go is... Backup all your data on your current system. Destroy your pool. Add the 5 additional drives. Create a RAIDZ2 or RAIDZ3 with all 10 drives. Before restoring your data, ensure the pool was created as you expected as the Volume Manager will often change the RAIDZ configuration once you select how many hard drives you want to use. It tries to "optimize" the selection however if you're running slow hard drives, this optimization isn't very valid and most folks won't even notice. Then Restore your data.

You can do it a few other ways but they have drawbacks.

This part has nothing to do with your question but may be a concern. One thing you didn't mention is if your using ECC RAM or how much. If you are not using ECC RAM then I would caution you to upgrade your system appropriately to ECC RAM and if you don't know the hazards just ask and a lot of people will chime in. And depending on your pool size ensure you have lots of RAM. Lastly ensure your power supply can handle the extra addition of these 5 drives, especially if you sleep the drives and they power up all at once.

Good Luck!

 

[fungus1487]

Hi guys thanks for the response. Some comments....

Im not actually noticing any performance issues currently.

I have 16GB of ECC RAM (as i said i planned ahead hardware wise)

Backup - destroy - create isnt really an option for me with 8tb of data.

As far as im aware extending the current volume means having a dependency on both sets of drives for new file writes, so for safety I may just have to go with a seperate volume on an independant share.

 

[joeschmuck]

If you have not noticed performance issues then you are not heavily using the NAS. That is good for you.
I think the separate volume of 5 drives is the best alternative. But you could EXTEND your current pool but the downside is a dual drive failure is a complete loss of all data. Having 10 drives really increases the risk but you retain the single pool name.

8TB of data is quite a bit but using a stack of DVD-R's might be a good thing to backup your data. If it's just movies then maybe you could backup the ones you don't want to rip again and remove the ones that you don't think you will watch again. Just a thought, good luck.

 

[fungus1487]

If you have not noticed performance issues then you are not heavily using the NAS. That is good for you.
I think the separate volume of 5 drives is the best alternative. But you could EXTEND your current pool but the downside is a dual drive failure is a complete loss of all data. Having 10 drives really increases the risk but you retain the single pool name.

8TB of data is quite a bit but using a stack of DVD-R's might be a good thing to backup your data. If it's just movies then maybe you could backup the ones you don't want to rip again and remove the ones that you don't think you will watch again. Just a thought, good luck.

Thanks, I may go this route as your backup is only as good as your backup of it ;)

Indeed the NAS isn't really stressed with a max of 2 users at any one time. However I still get around ~100MB/s read 80MB/s write to/from the NAS which I am more than happy with after playing around with pre-built solutions (which were terrible). The only time I see any problems is with multiple writes (4 or more at the same time) this seems to hit the CPU hard, which is expected.

I have also been considering http://forums.freenas.org/threads/new-plugin-available-crashplan.15347/ as a backup provider. I know it will be throttled, but my ISP doesnt throttle my connection. It would take a while to backup what I have but once done, incremental's should be small.

I priced this up on Amazon S3 and they dont even come near.

 

[TheWiz]

I also have same problem like u.

I have 4X 2TB WD Caviar Green (that i take from my first shit NAS WD 8TB ShareSpace),
and now i want to had 2X 4TB WD Caviar Green. 99%, 5.2TiB, 14.2GiB free.

I also don't know how to do it. Copy data, destroy and create new pool isn't an option for me to.

Other thing, what i need to do now so in future when i want to had more 4TB Caviar Green disk i don't had same problem.

 

[joeschmuck]

I'm sorry but it's kind of funny. These are some very large pools and I think you need to get honest and evaluate how important your data is. If the pool was to die, how bad would it be for you? Realize that if you have a single drive failure, a full 4TB drive will take a very long time to resilver, I am talking long time like over a day or more. If you have a second failure, poof! Data all gone, no recovery options unless you're wealthy.

Look at your options and choose. If you are going to continue to grow your pool, life is going to become complicated quickly.

@TheWiz,
One option you could choose is to replace one of your 2TB hard drives with a 4TB hard drive, let it resilver, verify the pool is good, then and only then replace the next drive. Repeat until you have replaced all four hard drives. Once done your pool will expand to the full 10.9TB of RAIDZ1 space, assuming you are running RAIDZ1. This is not for the light of heart because a failure of one of the old drives during this process means a loss of your entire pool, which is a good reason for RAIDZ2 use. It's an option.

 

[TheWiz]

If i have 4X 4TB = 16TB.
With Raid-Z i have 10.9TB
With Raid-Z2 i have 7.3TB
With Raid-Z3 i have 3.6TB. Crazy!
Right now i have 4X 2TB on Raid5, so i have 5.5GB.
My info is movies and series i like to save and watch. Not critical info. What you recommand me?

 

[joeschmuck]

Well technically you can't have a RAIDZ3 unless you have 5 drives. Keep in mind that ZFS is really designed for commercial use for data integrity, not home use. But I like it for home use, love the reliability aspect.

 

[Mr_N]

i'd buy another 2 drives, make a seperate volume with 6 using raidz2, move the data over to that volume, add the final disk and recreate the raidz as a 6 disk raidz2 vdev, adding it to the new pool :)

 

[joeschmuck]

i'd buy another 2 drives, make a seperate volume with 6 using raidz2, move the data over to that volume, add the final disk and recreate the raidz as a 6 disk raidz2 vdev, adding it to the new pool :)

First I'd say your message isn't very clear and also I believe that is bad advice as written. Adding a single drive to an already existing pool is unwise as a failure of that specific drive would cause the entire pool to be unrecoverable.

 

[Mr_N]

how is that not clear?

1) buy 2 more drives, put one aside
2) use the 5 new drives you have plus one of the extra bought in (1) to create another pool of 6xHDD in raidz2
3) transfer all data to new raidz2 pool created in (2)
4) destroy original raidz 5xHDD pool
5) use the second extra hdd from (1) and the 5 hdd's from (4) to create your new 6xHDD raidz2 vdev and add it to the new pool created in (2)
6) you now have the extra space you wanted, and you pool is more redundant

clearer?

 

[joeschmuck]

Yes but that was the basic idea back on my first posting but using the 5 drives the user had on hand and running RAIDZ1 as he wanted. I'm not sure if he has the capacity to run 12 drives. Either way these are good options.

 

[Mr_N]

yeah well he did say backup wasnt an option so hopefully he has room for 2 extra drives ;)