8 drive zpool config question

[namboy]

I've been reading all the documentation I can find - including the ZFS Storage Design Power Point - about the configuration of zdev's into zpools. I'm reusing an old IBM Intellistation workstation (nice power supply and lots of room) and I have 8 4Tb drives. None of the zdev configurations use 8 drives directly. Would it be correct to put 4 drives in zdev1, 4 drives in zdev2 and then put both of those into 1 zpool? If so, then my next question is what is the correct RAID format - raidz or raidz2?

 

[indy]

Using 8 drives in a raidz2 configuration is supposedly bad.
There are many forum posts and blogs that suggest using 2^n + p drives, the FreeNas documentation does as well.
However I could not find mention of this in Oracle's 'Recommended ZFS Storage Pool Practices'.
In fact this paper mentions 'Create a triple-parity RAID-Z (raidz3) configuration at 8 disks (5+3)' which would seem as unfavorable as possible.
Also the few benchmarks I could find do not reflect on the supposed disadvantage of odd drive numbers.

Of course its up to you to decide what to do, but with 8 drives you do not have a lot of options that use the available space efficiently.
I myself am currently purposely building a 8 drive raidz2 system for different reasons, its not up yet however.

 

[louisk]

I'm using 8x 2T drives as mirrors in a single pool. Works fine.

 

[indy]

Having 50% of the available disk-space 'wasted' on parity is a big trade-off imo.
Plus with raidz2 any two disks may fail.

 

[namboy]

I'm using 8x 2T drives as mirrors in a single pool. Works fine.

I too have another FreeNAS system that is using 8 3Tb drives all in zdev1 and 1 zpool, and it is running well. But what I was after - on this new box - was to build a more optimal configuration. When I built the existing 8x 3Tb box, FreeNAS warned me that it was not an optimal system. I guess the read/write may be slower because the blocks of data do not match-up evenly ..... ?

 

[louisk]

One person calls it wasted, another calls it insurance. It probably depends on what your data is worth, or the performance requirements you need to meet.

Best practice has probably changed since I looked it up last (2008ish). I've always gone with the "power of 2, and not more than 8" rule for a vdev. I also make sure I use a small SSD (32/64G is plenty, but hard to find anything smaller anymore) for the ZIL and a 32-128G SSD for the L2ARC. 32G SSDs are pretty cheap now, so that shouldn't be hard to do, and performance does go up a fair bit. Don't forget to get 2 for the ZIL and mirror them. L2ARC crashing slows things down, but won't be a deal breaker.

 

[indy]

Is Raid10 safer than raidz2 in an 8 drive configuration?

 

[Michael Wulff Nielsen]

Raid 10 would be 4 drives striped and then mirrored. So with Raid10 you could loose 4 drives and survive. With raidz2 you can loose two drives. But I consider the risk of loosing two drives between having resilvered one drive to be very unlikely.

But for larger pools you would probably want to go with raidz3.

You could also run on 7 disks with raidz2 and get 15TB and leave a single disk in the machine as a cold spare.

 

[louisk]

Not quite. With mirrors in a pool, you can lose up to 1 drive from any mirror vdev w/o losing data. With raidz2, you can lose up to 2 drives from the raidz2 vdev w/o losing data.

 

[joeschmuck]

@namboy,
You are fine to create a pool using 8x4T drives as a RAIDZ2. The GUI will flag this as not optimum simply because it doesn't meet the optimum configuration for maximum throughput (that formula mentioned a few posting earlier). If you're not building a high demand office server using enterprise drives, you will be fine and never notice that you can't open 30 files almost simultaneously (slight exaggeration).

I'm not sure why you need all that storage space, maybe it's for a large amount of snapshots so its good you factor that into the project up front. If the data is very important, like work related and you can't afford to lose it, RAIDZ3 would be a smart choice.

You should have enough advice now to figure out where to go from here.

 

[namboy]

@namboy,
You are fine to create a pool using 8x4T drives as a RAIDZ2. The GUI will flag this as not optimum simply because it doesn't meet the optimum configuration for maximum throughput (that formula mentioned a few posting earlier). If you're not building a high demand office server using enterprise drives, you will be fine and never notice that you can't open 30 files almost simultaneously (slight exaggeration).

I'm not sure why you need all that storage space, maybe it's for a large amount of snapshots so its good you factor that into the project up front. If the data is very important, like work related and you can't afford to lose it, RAIDZ3 would be a smart choice.

You should have enough advice now to figure out where to go from here.

****************************************************
Thanks for all the reply's guys. joeschmuck - it IS a office server (maybe not high demand ...) I have to store a lot of imagery - satellite imagery. Most files are 1Gb each, some much larger, so it's mostly streaming type of R&W, not a lot of random small file IO.

Which leads me to another question: the noobie Power Point seems clear that this work load would not benefit much form ZIL or L2ARC ssd's? I'm asking because I just want to make sure there is no other way to speed up R & W (other than RAM - I'm using 20Gb).

 

[joeschmuck]

A ZIL or L2ARC would not benefit you for what you are using the FreeNAS server for. You likely won't need more that 8GB RAM either, again based on what you are doing. You are writing and reading very large files, not thousands of small files and not accessed by a lot of people at the same time. However I would recommend 16GB RAM just because you never know and you already are using 20GB RAM so you have lots. There is no way to improve throughput speed based on more RAM or ZIL/L2ARC. The only way would be to use 10Gb Ethernet connections. If you transfer data over a Gb connection and you can write at or above 90MB/sec continuously (large file of at least 1GB) then you are doing very well.