Creating a RAIDZ2 with 10 drives is an OK setup?

[mhweb]

I'm about to build my first FreeNAS, and I was planning to buy 6 x 10TB HDD to create a RAIDZ2 storage pool, but when I mentioned it in another discussion, I was recommended to get 10 x 6TB drivers to create a RAIDZ2, which in theory it's going to give me more usable space.

However, I usually read that a storage pool is recommended to have no more than 8 drive. Is there anything wrong using 10 or more drives to create a storage pool? One of the reasons I want FreeNAS is to keep everything on a single redundant location.

 

[Heracles]

Hey MHWeb,

You sure can do it. Either a 6 drives RaidZ2 or a 10 drives RaidZ2, both options are good.

Things to keep in mind are :
Your pool will have the performance of as many drives as you have vDevs. So if you have a single vDev, no matter it is 6 or 10 drives, you will have the performance of a single drive.

Should you wish to do more IO intensive tasks like iSCSI, Raid-10 would be better With 10 drives, that would give you 5 vDevs, so 5 times more performance.

About capacity, you are way better to put too much from the beginning than facing problems later. Your pool's performance will degrade as your pool fills up. There are cases where you do not want your pool to be loaded at 50%, sometime even below 25%.

You did not mentioned what model you are looking for. Know that some drives are so bad that they are to be avoided at all cost. Some are just Ok and others are great and designed for NAS usage.

About redundancy, never forget to do your backup properly. No single FreeNAS server can offer a complete protection. See the 3 copy rules in my signature about that.

Have fun designing your system,

 

[mhweb]

@Heracles this is the hardware:

10x6TB WD Hard Drives 5400 (WD Red 6TB NAS Internal Hard Drive - 5400 RPM Class, SATA 6 GB/S, 256MB Cache, 3.5" - WD60EFAX)
CPU: Xeon E5-2680 v3
Cooler: Noctua NH-U12DX i4
RAM: 32GB (4x 8GB) SKhynix DDR4 Server RAM ECC
Mobo: SuperMicro X10SRL-F ATX
SAS: Genuine LSI 6Gbps SAS HBA LSI 9201-8i
PSU: Seasonic FOCUS GX-650, 650W 80+ Gold
Case: Rosewill RSV-L4412 - 4U (used)
SSD: boot drive

 

[mhweb]

@Heracles what would you recommended in terms of hdd brands and usable capacity over 40TB?

 

[Heracles]

Hi again,

The WD Red series you mentioned is part of the good ones and designed for the job. As for me, I bought Seagate IronWolf which is also good and designed for that.

To reach 40 TB out of 6TB drives, you need at least 7 of them as usable space. That means Raid-10 with 5 mirrors would not do it. To have more than 1 vDev with RaidZ2 would also lower you to 6 drives of usable space.

So 40TB out of 6TB drives, that would be a RaidZ2 or RaidZ3 10 drives vDevs.

 

[lopr]

Things to keep in mind are :
Your pool will have the performance of as many drives as you have vDevs.

Only IOPS, throughput will scale. For the IOPS the caching helps. I never came across any performance problems on SOHO use. But as Heracles said for iSCSI or heavy database use go mirror.

 

[Apollo]

WD RED are good but they lack responsiveness due to being 5400 RPM drives. IronWolf, which I have no experience with are I beleive 7200 RPM so they will be snappier.
2 Vdevs would be the better option for WD RED. The performance gain is noticeable.

 

[Johnnie Black]

WD60EFAX

Those are SMR and not recommended for FreeNAS, WD60EFRX are fine.

 

[mhweb]

@Johnnie Black How can you tell which one is SMR and which ones are PMR before buying them?

 

[mhweb]

@Apollo are the IronWolf noisy? The only reason I chose the 5400 apm is because I was told they're quieter. Are your 7200 drives loud?

 

[Apollo]

@mhweb I don't own any IronWolf drives so I can't say.
I have the Seagate Archives drive using SMR, and I don't know why they are not recommended as they are perfect from ZFS. They are used for receiving replication but I had them put to the test when I first got them.
I have a set of HGST NAS at 7200 RPM and they are a bit noisier than the WD RED due to the more aggressive head tracking motion.
For a server, this isn't really a problem unless you sleep next to it.
I would expect IronWolf to behave somewhat the same way.

 

[mhweb]

@Heracles Maybe I'm not understanding correctly, but you're saying that the pool will have the speed of one drive, but on online reports and videos show that a RAIDZ2 has higher performance than a single drive. (This video is an example https://www.youtube.com/watch?v=GGd-DiDnLpw)

 

[mhweb]

@Apollo I have small space, and I have people complaining when my current external HDD connected to router turn on at night for maintenance, so I want to keep noise to a minimum, or a least balance with performance, because I don't want to have a super slow NAS either.

 

[Apollo]

@Heracles Maybe I'm not understanding correctly, but you're saying that the pool will have the speed of one drive, but on online reports and videos show that a RAIDZ2 has higher performance than a single drive. (This video is an example https://www.youtube.com/watch?v=GGd-DiDnLpw)

I skimmed through the video quickly ( I watched it some while ago too) and the test setup isn't too clear. If he is only writing and reading one single file, this isn't pushing the system at all which makes his conclusion inconclusive and misleading.

If the drives are on proper dampening robber gromets, then the noise would be reduced. Beyond that I can't provide objective data.

 

[Heracles]

Hi,

Indeed, there are many scenarios you need to test to evaluate performance. RaidZ2 will speed up your access and be faster than Raid10 when reading a single continuous set of data. It is when you do random access that RaidZ2 starts to suck and Raid-10 will do much better.

iSCSI or databases will do random access. A plex media server will do long sequential reads. So what will you do ? When you don't know, safer to go with what is faster with random access. I also mentioned in the first place that this choice is to be based on the fact that you do or don't things like iSCSI.

 

[Johnnie Black]

@Johnnie Black How can you tell which one is SMR and which ones are PMR before buying them?

WD60EFRX is always CMR, WD60EFAX is always SMR.

 

[Jessep]

RaidZ2 is the same IOPS as a single drive, sequential reads/writes can be higher than a single drive.

 

[zamana]

I did exactly this in the recent past, but changed for a "raid10": 5 pairs of mirrors.

The reason: I can upgrade my storage "horizontally", by adding more pairs of mirrors, and/or "vertically", by replacing the existing disks with more capacity ones.

With a RAIDZ1, 2 or 3, you will be "stucked" with it, let's say... "forever", or until you decide to destroy everything and make it again, and restore the data from your backup.

In the end it's a matter of choice, but keep this in mind.

Regards.

 

[Chris Moore]

2 Vdevs would be the better option for WD RED. The performance gain is noticeable

Very true. I went to 2 vdevs for this reason.

 

[Chris Moore]

Are your 7200 drives loud?

There is a noticable difference and Western Digital drives tend to be louder than the Seagate drives.