Are two RAIDZ2 vdevs more performant than one RAIDZ3 vdev?

[Alan W. Smtih]

I'm preparing to build my second FreeNAS box to hold my photo archive. A bigger, badder version of my original 6 drive machine. After researching these forums, I've decided to use 4TB WD Red drives. I've narrowed my configuration plans down to either:

• 11 drives in a single RAIDZ3 vdev (11 drives with 3 for parity), or
• 12 drives split into two RAIDZ2 vdevs (6 drives each with 2 for parity)

Both of these configurations would produce 32 TB of storage.

• (6 drives - 2 parity) × 4TB per drive × 2 vdevs = 32 TB available storage
• (11 drives - 3 parity) × 4TB per drives x 1 vdev = 32 TB available storage

I'm leaning toward the 11 drive RAIDZ3 configuration. (One less drive to buy, case shopping will be easier, fans won't have to work quite as hard...)

My question either configuration offers a performance benefit. Would there be a difference in speed when reading/writing, say, a thousand 30MB image files at a time over gigabit ethernet?

---

Other background info if it matters:

• Probable motherboard: ASRock C2550D4I Mini ITX Server Motherboard
• Likely 32GB of RAM (unless 64 would make a huge difference)

The C2550D4I has twelve onboard SATA connections that the drives will be wired to. The controllers are:

• Intel C2550: 2 x SATA3 6.0Gbps
• Intel C2550: 4 x SATA2 3.0Gbps
• Marvell SE9172: 2 x SATA3 6.0Gbps
• Marvell SE9230: 4 x SATA3 6.0Gbps

I don't see those listed on the FreeBSD Hardware Compatibility List, but a review of the board on NewEgg says it's working fine with a FreeNAS install.

 

[cyberjock]

Performant on what what? I thought this had been covered about 10000 times already.

throughput is based on the number of disks. iops is based on the number of vdevs.

 

[SirMaster]

You will actually end up with more space with the two 6-disk RAIDZ2 config.

And yes more vdevs does provide more performance for workloads that need lots of iops. Though with your photo archive workload it shouldn't matter.

 

[Alan W. Smtih]

Performant on what what? I thought this had been covered about 10000 times already.

throughput is based on the number of disks. iops is based on the number of vdevs.

Howdy, cyberjock. My main questions was about throughput. I also had a secondary objective of posting some specifics as a type of sanity check. I'm sure this has been covered a ton, but with the tonnage of info on this site, I wasn't able to easily put my hands on it. (Or, probably better said, after hours of kicking around, I hit the exhaustion wall.)

When I searched for "raidz2 raidz3 speed" in the storage forum, only eight results showed up. Three of which were from the same thread and none of which appeared to answer my basic question. The good news is that this thread now shows up there as well. Hopefully that'll help answer the question for others who use the same search terms.

Also, thanks for putting together your guide. I don't usually pay attention to signatures so totally missed. Going through it is my next step.

 

[david kennedy]

ZFS is not a "high performance" file system, it was not one of the design goals (the actual goals are posted below). You can make it fast (add ram, L2ARC, etc) but it was designed to take consumer drives and build reliable pools with minimal effort.

Anyhow, as i am sure Cyberjock will post as well, you can easily saturate GIG-E with almost any hardware, so what is your goal?

Build a pool that accomplishes your actual NEED, not what you want. Do you want or need a fast pool for your archives?

• 12 drives split into two RAIDZ2 vdevs (6 drives each with 2 for parity)

I believe you actually have 4 parity drives in this situation (two pools, each raidz2 so two parity PER POOL or 4 in total)?

These posts tend to remind me of the "I need high grade encryption because" posts.

--
The following are the primary design goals for ZFS: (notice speed is not listed, except by adding SSD's)

• Simple: The goal is to remove the complexity of file system and volume manager administration—
tasks that have traditionally been costly, error prone, and not scalable to storage systems with
thousands of devices.
• Easy administration: A pooled storage design and a simple yet flexible command set make it easy
to provision and manage the storage environment.
• Data integrity: In petabyte storage configurations for which ZFS is aimed, device failure and
degradation are a given, so end-to-end data integrity and the ability to recover seamlessly and quickly
from component failures are a must.
• Ability to scale up or scale down: ZFS is a 128-bit file system with no practical limits on the
number of files, directories, and file systems, and no practical limits on the amount of physical
storage that can be addressed. Performance can be scaled up as well by putting applications on faster
solid-state drives(SSDs). Storage footprints can be scaled down 3X by using ZFS compression.

 

[Alan W. Smtih]

David,

You're totally right and I'm on the same page. I wasn't clear enough in my initial post. Keeping my data safe is my goal. I'm just wondering if I would see a difference by simply adding one more hard drive to jump from a single RAIDZ3 vdev to two RAIDZ2 vdevs.

 

[jgreco]

The two Z2's theoretically have more performance. If you have lots of people simultaneously accessing the service, lots of small files, or lots of IOPS needed, then that is definitely a consideration. The Z3 offers better protection, especially if you buy all 12 drives and leave one slotted in as a warm spare, so that you can command a rebuild without even having to touch the NAS.