Help determining best layout

[MadsRC]

I've recently aquired a 24 disk SuperMicro server, and want to play around with FreeNAS at home.

The last couple of days have been spent reading the wiki, this forum and watching videos and taking notes. My question is though, what would be the best layout?

My plan is to have one Volume/zpool, that is then split into datasets (As I will need a couple of "shares" for backup, files and media). From what I can read, a pool can consist of one or more stripped vdevs. This would also mean that if one vdev fails (Say a disk fails in a stripped vdev or 3 disks fail in a raidz2 vdev) the whole zpool is lost (If I'm correct?)

I have the following disks:
14 x 2TB drives
8 x 1TB drives
2 x 250GB drives

I was thinking something like:
6 x 2TB drives in RAIDZ2 + 1 spare
6 x 2TB drives in RAIDZ2 + 1 spare
7 x 1TB drives in RAIDZ2 + 1 spare

I don't plan on using the 2 x 250GB drives right now, as I'd either use them for larger disks, cache/log (I know, buy more RAM first!) or more spares.

Now, it's for home use, so I'd like a compromise between space and safety. From what I understand, I need to enable the autoreplace function for the spares, am I correct?

What are your thoughts on the above setup?

 

[cyberjock]

If I had all of those drives I'd probably do a 14 drive RAIDZ3 and an 8 drive RAIDZ2 in one zpool(but you could do 2 zpools if you wanted).

The 250GB drives won't work well for cache or log. You really need SSDs if you want a cache or log device to actually provide a benefit as those need high throughput and ultra low latency.

 

[MadsRC]

Thank you CyberJock! I really appreciate your input.

You wouldn't utilise the spare function?

I know the 250gb drives wouldn't do much for anything. Maybe I wasn't clear, but I don't plan to use the actual drives for anything. My plans are for the slots, with all new drives (which may be SSD's for cache/log or HDD's for storage)

 

[cyberjock]

I'm not a fan of the spare function for 2 reasons:

1. The spare function doesn't work with 8.3.1. The necessary features for ZFS to support it don't exist with FreeBSD 8.x. I'm also not sure if they exist in FreeBSD9/FreeNAS 9. Last I heard(and I could be wrong) but using spares was "supposed" to be in FreeBSD 9, but has been pushed back to FreeBSD 10. From what I've heard, it was promised in FreeBSD 8, then pushed back to FreeBSD 9, now pushed back to FreeBSD 10. So I'm not banking on seeing it anytime soon as it clearly is not trivial to implement and there's no guarantee it will work well(or at all).
2. I monitor my hard drives closely. I get nightly custom emails that tell me the SMART status of my drives and pool status. If a drive starts failing I want to make the decision to replace it myself, manually. There may be circumstances where drives are "failed" for whatever reason but the disks aren't bad. There may also be circumstances where you don't WANT to have a resilvering process auto-start. Like in the middle of a work day when high performance is needed at that particular time or if you are already running long SMART tests on hard drives. Running long SMART tests on hard drives while scrubbing/resilvering is a recipe for slowness.

If you are saying you have 2 free slots you have 2 options I'd consider:

1. Get a small Intel SSD(or whatever brand you trust) and install FreeNAS on that. USB sticks do work well and I've never had any fail and cause me problems but plenty of people have. Choosing to use an SSD, in my opinion, increases the reliability of your system. Less fuss and less things that can cause problems later. Keep the other slot free(see below).
2. Keep both slots free. If/when you decide to upgrade your hard drives you can do drive upgrades without degrading your array. This can be very handy if you have a disk or 2 that starts getting flaky and you want to replace the disks but don't want to risk deliberately degrading the array to do the replacement.

There's no reason to fill all of the slots just because they are there. I know its instinctive and you don't want to undersize your server either. But if you are sure that your zpool size will keep you happy with the drives you plan to use and have 2 slots free, that's great news! You might find you need a ZIL or cache in the future, and those could fill the slots just as easily. Personally, I prefer to keep a slot or 2 free for future endeavors just in case you want/need them someday. Nothing sucks more than having all of the slots filled and you deciding you need to add one disk and have no place to install it.

My system, I went big and went with a full 4U 24 drive case with all 24 ports connected to my server. I didn't use them all at first and it gave me a lot of versatility when i decided it was time to add some more disks.

 

[MadsRC]

Thank you so much for taking your time to write that.

I think I'll keep them free for the future. You never know what might come up.

 

[MadsRC]

Another question.

When creating the zpool, I get this warning when trying to add the raidz2 to my zpool consisting of a raidz3:

Code:

    You are trying to add a virtual device of type 'raidz2' in a pool that has a virtual device of type 'raidz3'
    You are trying to add a virtual device consisting of 8 device(s) in a pool that has a virtual device consisted of 14 device(s)

I guess I can force it (Which I did), but does it have any downsides? Why the warning? Is my data less secure this way?

 

[gpsguy]

The warning is to keep user's from making irreversible mistakes using the GUI.

All to often, users think they can expand their RAIDz2 pool with a single drive and older versions of FN allowed them to do it. While the first vdev had 2 parity drives, the second vdev had none. If the latter failed, they loose the entire pool.

In your case, with RAIDz3 and RAIDz2 it's probably an acceptable amount of risk.

Why the warning? Is my data less secure this way?

 

[MadsRC]

Thank you VERY much.