Replacing RAIDZ2, want a second look

[auralsun]

Hello,

I have an 8-bay NAS currently with 6 WD Red 4TBs installed in RAIDZ2, ~16TB available storage. Pool was created w/ FreeNAS in 2010, replaced CPU/RAM/MOBO/PSU/etc once since then and 2 failed HDDs. Never thought I'd run out of space but the time has finally come, getting 80% storage notifications system is currently sitting at 81%.

Since upgrading RAIDZ2 pools is kind of a pain I am leaning towards mirrored vdevs with much larger drives. Currently looking at adding 2 WD Ultrastar DC HC550 18TB as mirrored vdev in the two vacant bays, transferring data from the RAIDZ2 pool to new pool, removing the 6 original HDDs, adding 2-4 more 18TB drives later on to expand pool.

Just looking for a bit of feedback before proceeding w/ big HDDs purchase:

1. Does this generally sound like a solid plan? Any pitfalls I'm not considering?
2. Anything in particular to expect when using giant HDDs with TrueNAS beyond longer resilver times and making sure to upgrade RAM to accommodate the pool?

Thanks a ton!!

 

[sretalla]

1. Does this generally sound like a solid plan? Any pitfalls I'm not considering?

Depends if you're focused on cost or throughput... If you're not, it's fine. You will potentially get a little less throughput (but probably more than Gigabit anyway) from one or two mirrors compared to RAIDZ2, but you will get more IOPS, so that's maybe a plus.

2. Anything in particular to expect when using giant HDDs with TrueNAS beyond longer resilver times and making sure to upgrade RAM to accommodate the pool?

More RAM is always better, but you don't necessarily need to upgrade it unless your stored data increases... empty disk capacity isn't really costing you any RAM.

For sure resilvers on larger drives are longer, but Mirrors resilver faster than RAIDZ, so maybe an overall speed gain in resilvering.

What you will be adding is a risk that 2 disks failing will kill your entire pool (not the case with RAIDZ2)... see below, Red is Pool loss, Yellow is Degraded.

 

[Arwen]

One way to mitigate the 2 way Mirror risk, is to re-use the existing 6 x 4TB disks as a backup. Change them to a RAID-Z1 or even just a striped pool when the RAID-Z1 gets over 80% used. Since it would be an on-line backup, easy to have it replicate from the 2 way Mirrored pool.

Of course, this should not be your only backup...

And when you finally need to grow your Mirrored pool, you would have to remove some of the 4TB disks. Thus, removing this backup. (But, the 4TB disks could have started failing by then...)

 

[auralsun]

What you will be adding is a risk that 2 disks failing will kill your entire pool (not the case with RAIDZ2)... see below, Red is Pool loss, Yellow is Degraded.

View attachment 52315

Hey, thanks a lot! In light of your graphic I'm reconsidering upgrade path. To make sure I understand, basically with mirror vdev if the wrong 2 disks (disk1 in mirror1, disk1 in mirror2) fail then pool is toast, in order to have the similar safety as RAIDZ2 would need triple mirror?

Since my current chassis only supports 8-bay I'm considering upgrading the RAIDZ2 instead... wouldn't be possible to have a 3-3-3 mirror without adding more bays. In this case I replace each drive individually, resilver, when all drives in pool are replaced with larger capacity I can expand pool size?

 

[auralsun]

Is Backblaze drive data still the best source for gauging drive model reliability?

 

[jgreco]

Is Backblaze drive data still the best source for gauging drive model reliability?

It's funny, you say that as though you think that it was the best source for gauging drive model reliability in the past.

There isn't much public data out there one way or another. It certainly isn't the best source, but it is one of the few public sources that can give you some idea about very particular drive models.

 

[auralsun]

It's funny, you say that as though you think that it was the best source for gauging drive model reliability in the past.

There isn't much public data out there one way or another. It certainly isn't the best source, but it is one of the few public sources that can give you some idea about very particular drive models.

I use it for red flags for drive models with very high failure rates, not sure if there are other sources that publish similar data. I seem to remember finding somewhere indicating the 16-18TB Ultrastar drives are pretty reliable but can't find data with drives this large now, reason I ask

 

[jgreco]

Fair enough, it's always good to avoid known problem drives. Unfortunately, in this business, past performance is not a predictor of future success. Alas.

 

[Etorix]

Since my current chassis only supports 8-bay I'm considering upgrading the RAIDZ2 instead... wouldn't be possible to have a 3-3-3 mirror without adding more bays. In this case I replace each drive individually, resilver, when all drives in pool are replaced with larger capacity I can expand pool size?

With 8 bays, 6 of which are used, you may even replace one or two drives at a time, leaving the old drives in place so you have full redundancy all along. Only remove the old drive(s) when resilver if complete.
If the right flag is set, which should be the case by default, the pool will automatically register the larger capacity upon replacing the 6th drive.