Recommended RAID setup, price-wise?

[the_jest]

I'm not sure if this question properly belongs here or in the hardware forum. But: I'm using my box mainly for local media/file storage and similar stuff for a small number of users--not running VMs (well, I have one small VM running PiHole, but I mean I'm not using it to back enterprise-level database servers or anything). In my current setup, I have four 4TB WD Red drives set up as two twin mirrors, for a total of 8TB of storage.

I'm not in urgent need of more space, but I probably will need to expand things in the medium-term future, and I see that Amazon currently has the 6TB Seagate Iron Wolf 7200 RPM drives for $148. This seems pretty good to me, and I thought it might make sense to consider doing something with these.

Would a three-drive RAID Z/1 setup make sense for my needs? This would give around 12 TB of storage. A four-drive setup would seem like overkill for me (I do have several backups of the entire NAS). Is there something else I should consider? Poking around this does look like a good price, but maybe there's something else I'm missing. Also, if I did this, what should I do with the existing drives--can I reconfigure them into something functional, or would it end up slowing things down too much? I do have the space in my box for eight drives.

 

[PhiloEpisteme]

Amazon currently has the 6TB Seagate Iron Wolf 7200 RPM drives for $148.

You may end up with higher temperatures with these 7200rpm drives. I've got a mix in my system with a lot of fans and my 7200rpm drives always run hotter such that I keep my fans going most of the time. For this reason lots of folks opt for the lower RPM drives following the school of thought that colder drives last longer. There are some interesting web articles on this that might be worth checking out. In fact, here is one suggesting humidity and not temperature are the real culprits for drive failure.

Would a three-drive RAID Z/1 setup make sense for my needs?

Lots of folks recommend against RAIDZ1 because of worries about an unrecoverable read error while resilvering a drive. Basically the issues is summarized as when you lost 1 disk in RAIDZ1 your data is still in tact but you need to replace that lost drive via resilvering. Because you're down a drive you have no parity data to work with and if there is an unrecoverable read error during resilerving you may end up with the effected file being corrupted. How big of an issue this really is depends on the error rate of the drives you use, their size, and number. This article sums it up nicely. Of course, in the end it depends on what data you're storing and your risk tolerance.

 

[myoung]

The cheapest and easiest option is to get two drives and add another mirror.

 

[the_jest]

You may end up with higher temperatures with these 7200rpm drives.

But more speed! Granted, I don't actually need more speed, but I'm going by what's available at a good price.

Lots of folks recommend against RAIDZ1[....]

What do those folks recommend instead?

The cheapest and easiest option is to get two drives and add another mirror.

Sure, if the only thing I cared about was extra space, but then I'd have so much random stuff going on, and this pair would be 7200 RPM vs. the 5400 WD Reds.... I'd like to be consistent about what I'm doing: "now my main storage is in [some format]", rather than "Now I have a mashup of a bunch of random things."

I see that there's a big Prime Day sale on the WD Elements 10TB external drive, which apparently contains the WD Red inside, so I could also get a few of those; of course it raises the same issues: Three in a RAID Z1? A mirrored pair? And with any such setup, what do I do with my current disks?

Anyway I just wanted to take advantage of good pricing now, I'm just not sure how to do it.

Thanks for any further thoughts from anyone!

 

[PhiloEpisteme]

But more speed! Granted, I don't actually need more speed, but I'm going by what's available at a good price.

Generally you'll get more performance out of well designed pools, more RAM, and SLOG + L2ARC devices where appropriate.

What do those folks recommend instead?

If you're not using mirrors folks would recommend RAIDZ2 or RAIDZ3 instead. RAIDZ2 is a nice trade off because it will tolerate 2 disk failures in a pool. In most cases you experience a single disk failure and replace the drive, if an unrecoverable error occurs during resilvering you'll have parity still to prevent data corruption.

What case are you using by the way?

 

[the_jest]

What case are you using by the way?

Silverstone DS380. Very happy with it.

 

[myoung]

Generally you'll get more performance out of well designed pools, more RAM, and SLOG + L2ARC devices where appropriate.

To piggy back on this, if you are switching from a pool of mirrors to a raidz based pool, you are likely to see a performance drop regardless of how many RPMs your disks have.

 

[myoung]

Sure, if the only thing I cared about was extra space, but then I'd have so much random stuff going on, and this pair would be 7200 RPM vs. the 5400 WD Reds.... I'd like to be consistent about what I'm doing: "now my main storage is in [some format]", rather than "Now I have a mashup of a bunch of random things."

I'm not sure what you think you mean. While using similar disks is ideal, there is no problen with having different vdevs made of different types of disks. You are making way to much of a deal out of RPMs and over-complicating your upgrade process. Why do you think three mirrors will be more random/complicated than the two mirrors you have now?

 

[the_jest]

I'm not sure what you think you mean. While using similar disks is ideal, there is no problen with having different vdevs made of different types of disks. You are making way to much of a deal out of RPMs and over-complicating your upgrade process. Why do you think three mirrors will be more random/complicated than the two mirrors you have now?

Well, for one thing, all my disks are identical right now, and I have two mirrors, one just backing up the other. (I realize I may be using the wrong terminology, and I'm away from the box now so I can't check it, but I believe I have two pairs of striped disks, one pair mirroring the other.) So this seems pretty straightforward to me. If I add a third pair of made up of different-sized and faster disks, I don't know how I would use it--I'd need four disks to stripe in the same way, or I could have just the two mirroring each other, but either way it would be different from what I have now in a way that would seem to be confusing. Would I put different things on the new ones? Etc.

 

[PhiloEpisteme]

Well, for one thing, all my disks are identical right now, and I have two mirrors, one just backing up the other. (I realize I may be using the wrong terminology, and I'm away from the box now so I can't check it, but I believe I have two pairs of striped disks, one pair mirroring the other.)

I may be misunderstanding what you're saying or you may be misunderstanding your pool and how vdevs relate to pools. My understanding was that you have 1 pool made up of 2 vdevs. Each of those vdevs is a mirror vdev composed of 2 disks. When set up in this way the vdevs do not mirror each other, instead the disks within each vdev mirror each other. Consider the following

Code:

pool:
  vdev1 (mirror):
    disk1
    disk2
  vdev2 (mirror):
    disk3
    disk4

In the above example, if this accurately reflects your setup, then vdev1 does NOT mirror vdev2, instead disk1 and disk2 are identical and disk3 and disk4 are identical. disk1 and disk2 are mirrors of each other, as are disk3 and disk4.

ZFS exposes pools for use as storage. Pools are composed of vdevs. Vdevs are composed of disks. You lose your pool when you lose a single vdev, thus redundancy is very important at the vdev level. Furthermore, because zfs exposes pools for storage you don't get to pick which vdev your data goes on, zfs spreads the data across available vdevs in a given pool. If you add a new vdev to a pool you still store your data at the pool level, zfs handles how it spreads data across vdevs.

If I add a third pair of made up of different-sized and faster disks, I don't know how I would use it--I'd need four disks to stripe in the same way, or I could have just the two mirroring each other, but either way it would be different from what I have now in a way that would seem to be confusing. Would I put different things on the new ones? Etc.

You do not have to add a vdev of matching disk size and speeds as the other vdevs in your pool. You could purchase 4 10TB disks and create two mirorr vdevs out of them and add those vdevs to your pool just fine. I think what seems confusing is a misunderstanding of how pools and vdevs work.

Your performance can be affected by a slow vdev. For this reason folks generally advice that you stick with similar vdevs within the same pool such that you get similar performance across vdevs. If you have a pool made up of mirror vdevs you'll get the best performance if you stick to the same style if you add a vdev to the pool. Given this, I would say generally that if you're planning to add a vdev to your existing pool that you stick with mirrored vdevs since that is what you already have.

Regarding performance at it relates to the rpm of a given drive consider that ZFS was designed for use with massive storage disks and attempts to be as fast as possible despite that. Whether you're using 5400rpm, 5900rpm, 7200rpm, 10krpm, 15krpm, etc drives if you're using spinning disks the IO on those disks will be relatively slow as compared to the other parts of your system. Thus ZFS has features such as the ARC and SLOG which allow you to improve performance of reads and sync writes respectively by trying to make intelligent use of RAM and faster but smaller and more expensive storage from SSDs and the like. If you do a quick search around the forums you'll see plenty of folks talking about trying to saturate their network connection using 5400rpm drives.

I hope this clears some things up. Looking back what I wrote I realized I rambled somewhat. Happy to elaborate on anything specific. Also happy to be corrected by folks if I've gotten anything wrong.