Pool size

[Nick Howard]

My hardware should arrive this week and I've started looking at how best to configure my system. I'll have 8 x 2tb drives and I had initially thought the best setup would be to create a Raidz2 setup. (6+2).

However reading a little more into this I keep seeing this power of 2 rule crop up and I'm wondering if I should add another drive for a raidz3 configuration (6+3) I do have a couple of spare drives so could do this. The only problem I'd have is that the Fractal design case only has 8 drive bays but I could easily fit one in the 5.25 bays I suppose. I'm also wondering if two separate pools of raidz (3+1) would be a good alternative.

Looking for some thoughts on the best route to take.

 

[Ericloewe]

RADIZ1 is to be avoided.

Ideally, RAIDZ2 would have 4, 6 or 10 drives. However, if you're using the server in a home environment with average workloads, non-optimal configurations may be good enough.

Assuming you want 12TB (well, 12*10^12 bytes, which is less than 12*2^40 bytes), the simplest option would be to use a RAIDZ2 with 6 3TB drives. The 3TB WD Reds, for instance, are currently the cheapest capacity in their family.

 

[Nick Howard]

Thanks Eric, When we talk about optimal configurations are we purely talking about the % of space lost to parity as opposed to the performance or both? If I'm reading this correctly the difference between using an 8 disk raidz2 config to a 10 disk raidz2 config i'd get the same performance but lose slightly more space to parity.

This example below confuses me a little as surely the 10 disk setup would have 20% lost to parity not 25% as indicated? or am I missing something?

 

[danb35]

For performance reasons which I don't fully understand, it's optimal to size RAIDZ vdevs at 2^n disks + parity, but not more than 11 disks total. This means a vdev should consist of 2, 4, or 8 disks, plus 1 (RAIDZ1), 2 (RAIDZ2), or 3 (RAIDZ3) disks for parity.

 

[Ericloewe]

Thanks Eric, When we talk about optimal configurations are we purely talking about the % of space lost to parity as opposed to the performance or both? If I'm reading this correctly the difference between using an 8 disk raidz2 config to a 10 disk raidz2 config i'd get the same performance but lose slightly more space to parity.

This example below confuses me a little as surely the 10 disk setup would have 20% lost to parity not 25% as indicated? or am I missing something?

As far as I know, the problem is as follows:

With non-power of two numbers of disks (other than 2, 4 or 8 + parity disks), ZFS sectors end up not being aligned with the disks' sectors (as the individual chunks that the disks receive are not multiples of 4K), which means more sectors have to be read, reducing throughput and increasing latency. Space losses should be small enough not to matter (less than 1 MB), especially when considering Swap and other reserved spaces.

The reduced performance should not significantly affect sequential reads/writes, but random accesses will suffer tremendously.

 

[Nick Howard]

That certainly makes sense, I guess I'll just throw another two drives into the top 5.25 bays and go with the 8+2 config.

 

[dlavigne]

I asked one of the ZFS creators about this and he created this blog post as the answer: http://blog.delphix.com/matt/2014/06/06/zfs-stripe-width/. We'll be updating the docs and Volume Manager accordingly for 9.3.

 

[cyberjock]

Yeah, and there's been discussions on this. I wouldn't change the manual just yet. There's MUCH more to this than it looks like on the surface.