Need Recommendations for setting up Storage

[imikejackson]

I am in the process of building out a FreeNAS 8.x server. I have all the hardware and will be putting it together this weekend. I have a quad core AMD Proc, 8 GB RAM, 6 SATA ports on the Motherboard and a case capable of holding at least 6 drives. 8 possibly. Now the question. I will be using 4x1TB drives for the initial storage.After a bad experience with another RAID device where the actual data structures got corrupted so the whole RAID volume got corrupted I would like to set this up with surviving a drive failure as the top priority. I have read and read and read many posts about ZFS (vdevs, pools, RAIDZ(1, 2, 3)), resilvering times, rebuilding RAIDZs and write performance. My last RAID was hooked up over USB so having this one over Gigabit ethernet should be much faster. The use of the NAS is to hold critical data that can not be lost. I also plan to run either a Time Machine sparse disk image or setup some sort of system to rsync my home folder to the NAS, then setup the NAS to snap shot the first rsync destination to have a "Time Machine" like backup.
I can not decide if I should have 2 x 1TB Mirror vdevs then combine both mirror sets into a ZFS Pool (I think I should get about 2TB of total storage capacity) or go with a RAIDZ2 of all 4 drives (which I think I should still get about 2 TB of storage). I could possibly purchase another pair of 1TB drives to get another mirror set or have a 6 drive RAIDZ2 system. All the drives are of different ages. Some are fairly new (<6months old) and some are about 2 years old.

My analysis is: 2x (2x1TB Mirror vdevs) : I could loose a drive in each mirror and be "safe". Loosing both drives in a mirror would definitely loose that entire vdev and data. Would I loose the entire pool? I also understand that I can expand the ZPool by creating a new vdev with another pair of drives in the future.

Single RAIDZ2 (4x1TB): I could lose any 2 drives and the data and the Pool is Safe? But I think the write performance may be less (how much I don't know) than the first option. I also understand that once the 4 drive vdev is created I can NOT expand that vdev with more drives. I would have to start a whole new vdev to add more storage space which would have to be at least another 4 drive vdev in RAIDZ2 in order to keep the redundancy of the ZPool.

Any practical experience from "production" systems (home or data center) would be greatly appreciated. I lost about 1 TB of data back in January which while not catastrophic did result in the loss of some old data that would have been nice to work with in future projects.

Thanks Again,
Mike Jackson (www.bluequartz.net)

 

[Joshua Parker Ruehlig]

if you ever lose all then drives in a vdev your lose the whole zpool. You have a very good understanding of zfs, looks like you did your research.

Though I don't care enough about my data to go from RAIDZ/2 to mirrors I have been wrestling with the though in my head.
Yes RAIDZ2 does have the advantage of loosing any 2 drives, mirrors have the advantage of less CPU overhead and faster recovery from a disk loss, performance wise mirrors win as well (though with your CPU I bet not by much). The answer really depends on your situation, I guess what we'd need is some longterm performance/reliability studies on zfs.

The best bet is off site backup of critical information. The rsync(over ssh) + zfs snapshot solution is what I have been doing and it works great. I even have an offsite location now (my sister's pfsense box I built for her from a p4 machine) though I don't backup stuff like movies and such, I guess it comes down to importance vs disk space.

 

[imikejackson]

I agree off site duplication of the data adds additional data redundancy and for some of the smaller parts of the data I have copies at home on a Drobo system (no comments on the Drobo. Let's stay on topic ). One of the areas that I can not seem to understand is how long a resilvering will take for a RAIDZ2 versus a ZPool spread over vdevs of mirrored drives? In some instances I see "weeks" in other instances I see "Days". I can handle "days" if I need to, "Weeks" is unacceptable.

So it basically comes down to what is the probability of loosing 2 drives where both drives belong to the same vdev mirror set. If happens I would loose the entire ZPool which is BAD. If I do a RAIDZ2 and I loose 2 drives then the data is still available but the resilvering process may take a much longer period of time compared to a ZFS mirrored vdev.

Maybe I should splurge for another pair of drives (1TB) and make a 6 drive RAIDZ2 which would give me a bit more storage room to start with considering I have about 1.75 TB worth of data to transfer when this thing is up and running.


The Specs of the system are:
AMD FX 4100 Black 3.6GHz/12 Cache
AsusTek MB (6 SATA Ports)
8GB DDR3-1600 RAM
Onboard SATA controller

Thanks again for any help or guidance.
Mike Jackson

 

[paleoN]

In some instances I see "weeks" in other instances I see "Days". I can handle "days" if I need to, "Weeks" is unacceptable.

A pool of striped mirrors with N vdevs take as long to resilver as a single mirrored pool. RAIDZ/2 pools take longer to resilver with each additional disk, similar to hardware RAID 5/6. With your hardware, the # of disks you are using and the amount of data, < 4TB, I don't see how you would manage to make it to weeks to resilver no matter what you did.

Maybe I should splurge for another pair of drives (1TB) and make a 6 drive RAIDZ2 which would give me a bit more storage room to start with considering I have about 1.75 TB worth of data to transfer when this thing is up and running.

4 1TB drives in striped mirrors, 2 vdevs, gives you 2TB raw. 4 1TB drives in a RAIDZ2 pool gives you 2TB raw. 6 1TB drives in striped mirrors, 3 vdevs, gives you 3TB raw. 6 1TB drives in a RAIDZ2 pool gives you 4TB raw.

A 6 drive RAIDZ2 pool sounds like a plan for you. Though you are going to be using about half your capacity all at once. Have you accounted for how much space you will need in the future?

 

[imikejackson]

I think as it stands now if I can pull together another pair of drives to get to 6 drives then that gives me about 2x raw storage over what I have now. Based on my expected workload that should keep me going for another year or so.

Thanks for the recommendations

Mike Jackson