Best Layout?

[brando56894]

I'm brand new to FreeNAS and ZFS, I've been bouncing around ideas of different layouts in my head but I figured I'd submit to the ZFS gurus out there to help me out. My NAS will mainly be for downloading and sharing multimedia.

After reading through a bunch of stuff about ZFS, I think I have come down to my final layout:

Should I be using RAID 0, RAIDZ2 or RAID 10 for my Multimedia storage? I would like the best read speeds possible. The lone 4 TB HGST drive will be added into the Multimedia array once I get another one since I currently have two sets of mirrored 3 TB green drives. Eventually these will all be replaced with HGST NAS drives, just don't have the money to do so yet.

Also I'm not sure if this is actually considered a RAID 10 array since it seems there's some confusion regarding how one creates a RAID 10 array. When I set up my volume I selected the 4 3 TB drives in a 2x2 mirror configuration, and when I look at the Volume Status it shows me two sets of mirrors, so is this actually a correct RAID 10 array?

I don't really care about the stuff on the 2x1 TB striped drives because those will be for temporary data from SABnzbd and Transmission, once stuff is finished downloading/post-processing it will be moved to the Multimedia volume (This was formerly a RAID 5 array in my Linux PC). My jails and my system dataset are on a pair of mirrored Western Digital 75 GB Raptors (10,000 RPM), which I plan on replacing with a pair of mirrored 80 GB SSDs. I currently have FreeNAS running off of a 32 GB Sandisk flash drive, which I plan on replacing with a 16 GB SLC SATA DOM (good idea? It's only like $75). Also would a ZIL help since I will be doing a fair amount (but not constant) writing? I know to not even think about an L2ARC until I max out my RAM.

System Specs
Asrock Rack C2750D4I w/Intel Avaton 8 core Atom @ 2.4 GHz
2x8 GB DDR3 ECC RAM (Supports up to 64 GB)

 

[Ericloewe]

You're using GbE, I assume. In that case, for general file-sharing, even RAIDZ3 is plenty fast (capable of saturating GbE).

(Second image is broken, btw)

If you value your data, go with RAIDZ2. If it's absolutely expendable, stripes. If it's valuable, but a loss is a "meh" kind of event, RAIDZ1 can also work (do expect to lose the thing, though).

Instead of spending crazy amount of cache on a DOM, either buy a real SSD for less cash (and more, faster storage!) or two OR just buy a second 32GB USB drive and mirror the boot drive.

Unless you're doing sync writes, an SLOG device will do absolutely nothing.

 

[Ericloewe]

Also, those things you called "vdevs" are actually pools.

The left and right ones have a single vdev (RAIDZ2 and mirrored, respectively) and the middle one has two (single drive).

 

[brando56894]

Thanks for all the info! The box actually has 2x10G Ethernet but my switch and router don't support that so I'm just using a single 1G connection. Looks like I'll probably be going with RAIDZ2 then since it sucks having to re-download terabytes worth of data (I've had to do that a few times already since my other NAS systems have failed in the past) and it's less expensive to add/replace drives in the future compared to RAID 0/10 (I fixed the image btw, stupid Google Drive!).

I guess I could forgo getting a DOM since I already have another 32 GB flash drive handy, the question is how much more would it benefit from striped flash drives compared to 1 or 2 "real" SSDs (either SLC or MLC)? The flash drives are capped at 40 MB/sec whereas the SSDs can reach 200-400 MB/sec. In order to mirror the boot drives I can just dump a raw image of the boot drive using dd and then write it to the new one, correct?

Thanks for correcting the terminology, I've been trying to wrap my head around the concept of pools and vdevs and all that, it's tough compared to a simple RAID array in Linux via mdadm hahaha

 

[Ericloewe]

There's no point in going with fast and expensive boot devices. You won't be booting regularly enough for the small difference to matter.

To mirror the drives, you have them both connected while installing FreeNAS and there should be an option to mirror them.
There's also an option to do so later from the GUI.
What you suggested, however, will not work.
In fact, it didn't work back in the 9.2 and older days, for reasons unknown (and since rendered irrelevant by the ZFS boot partition).

As for the now fixed image, that's two vdevs of two mirrored drives. Vdevs are always striped within pools (though FreeNAS will manage things a little more intelligently than strict 50/50). So, "RAID10"

 

[brando56894]

Ah ok, so nothing on the flashdrive is accessed once the system is up and running? I'm assuming the option you're talking about is Clone under System -> Boot correct? I wasn't sure if that would mirror the boot devices or just create a fallback in case the primary one failed.

 

[Ericloewe]

Ah ok, so nothing on the flashdrive is accessed once the system is up and running? I'm assuming the option you're talking about is Clone under System -> Boot correct? I wasn't sure if that would mirror the boot devices or just create a fallback in case the primary one failed.

Not quite, but the amount of IO is small. Settings and stuff, mostly. The .system dataset keeps the rest.

I haven't moved to 9.3 yet, so I don't know what the option is, but that sounds right.

Mirroring the boot device allows ZFS to fix potential errors (and believe, we've been seeing a lot of them on boot pools) and maintain proper availability. Without the mirror, you know something's wrong, but you end up having to reinstall everything and restoring the config from backup. The mirror is less of a hassle.

 

[Fraoch]

I'm assuming the option you're talking about is Clone under System -> Boot correct? I wasn't sure if that would mirror the boot devices or just create a fallback in case the primary one failed.

Actually you should follow this procedure:

http://doc.freenas.org/9.3/freenas_system.html#mirroring-the-boot-device

 

[brando56894]

Cool, thanks for the info.