Build: 13.6TB raidZ + one hot swap

[asw2012]

I've been looking into a NAS solution for my home for over a year and it looks like I've found my solution in FreeNAS because all the pre-built NAS systems I've been looking at are either way too pricey or too constricted. FreeNAS has so much to play with, its unimaginable. The past month, between scouring these forums and learning on a test rig I threw together from old parts, I think I've got a good grasp on hardware requirements.

I've got a ton of DVD's (1200+) varying from single layer to dual layer, and I want to back them up and possibly stream them throughout my house, also have a ton of music + backup data from my computers + use as a FTP + use Bit torrent - you get the idea. Going to build this NAS for the future, don't really want to run out of space anytime soon.

Setting up for RAIDz (6, 3TB Drives) + 1 spare + 30GIG SSD for a L2ARC

Please feel free to toss in your ideas in here for the hardware, in the next 2 weeks I'm going to pull the trigger

Fractal Arc Mini Computer Case

ASRock AMD Motherboard

AMD 3.4 Dual Core 65W CPU

16GB Corsair DDR3 (16GB is max this mobo can handle)

7pc. WD Green 3TB SATA 3 (6 + 1 Spare)

Sandisk 32GB SSD (L2ARC Drive)

PCIe to SATA 3 add-in card (will plug the hot swap and L2ARC SSD here)

USB 3.0 adapter - plug USB 3.0 Flash Drive directly into, OS will reside on this drive

Intel Gigabit Server Card - not going to rely on the mobo ethernet.

2pc. SATA 5.25 Rack mounting - 2 drives going into the 5.25 bays

any thoughts on the above? Would like some feedback from you all please!

thanks

 

[survive]

Hi asw2012,

Ok, here are my thoughts:

Check & be sure FreeBSD 8.3 will identify the chipset on your selected board. The A75 chipset is pretty new so FreeNAS might not recognize the PCI ID's...confirm before you buy.

I would urge you to consider doing raidz2 over raidz with 3TB drives. If you can I'd consider cutting back a little & just doing a 6 drive raidz2 or go a little bigger on the case, get an extra drive & do an 8 drive raidz2.

Note: Gigabyte makes a board with 8 SATA ports, it's ~$45.00 more than the AsRock, but you don't need to spend $30 for the Syba add-in controller, personally it would be worth it to me to avoid the Silicone Image controller alone.

The NIC is a bit overkill, why not just get an Intel "CT" card? Maybe throw the difference at a stand alone media player (check out the Raspberry Pi with OPENelec!) so you don't even need to deal with streaming?

I'd bet you won't actually need the L2ARC drive. I would try it first without it & see how it goes.

I'd forget using USB3 for the boot drive, you don't really need the extra speed and I've found that more often than not disabling USB3 support fixes many (odd) problems booting. Confirm this in the forums, but I'd spring for a good, fast, highly rated USB2 key myself.

Keep in mind that FreeNAS doesn't really do hot swap & spares. There's no reason you can't have an extra drive installed in the case that you can use to replace in when you have a failure (great for when the filer is a 100 miles away or something) but you still need to manually do the replacement.

-Will

 

[asw2012]

So, with the 3TB drives, you would tend to say there's a higher probability for a 2x drive failure? I am storing most to all my important information here, I do see your point with the added raidz2 redundancy. 8x 3TB drives with the raidz2 nets 16.4TB usable storage (plenty).

I was not aware the usb 3.0 drive caused odd problems. I'll switch to the 2.0.

I've experienced weird things with the silicon image controllers before but thats with my windows computers. A different situation with a NAS box... I'll omit it, and apply that money towards another 8-port sata mobo. Can't seem to find the Gigabyte motherboard you are referring to though. Is it based of an older AMD chipset, or is it Intel based?

I'll also try W/O the L2ARC SSD, save a few $$ there too

if you think the lesser intel nic would be good enough for my needs, then I'll use it: cheaper too!

thanks for the input so far.

 

[survive]

Hi asw2012,

I'm sorry....I forgot to include the link:

http://www.newegg.com/Product/Product.aspx?Item=N82E16813128572

If you are going to go 8 drives it might be worth going to a proper 8-bay case...take a look at the Fractal R4, should be $10 more than your selected case making it a wash since you wouldn't need the 2 adapters. You could also open up your board selection to include regular ATX boards.

There's nothing more stressful to your storage than a rebuild....that's the time when if somethings going to fail, it will fail. A rebuild with a fairly full 3TB drive can take a day or more so keep that in mind.

-Will

 

[asw2012]

Well, the hardware is on its way.

I went with your suggestion on a RAIDz2 array because I intend to use this as a backup + storage. To lose only the 3TB over RAIDz is worth the peace of mind if something totally whacko goes wrong, especially when storage is cheap.

Fractal Design Define R4
GIGABYTE GA-F2A85XM-D3H
AMD A4-5300 Trinity 3.4GHz
16GB Corsair DDR3
8 3TB WD Green drives
USB 2.0 internal header

:)

 

[asw2012]

Just an update:
Been running this for a little over a month now, no issues with the ZFS setup. I've got 15.2TB worth of space from the 8x3TB in RAIDz2. Performance is excellent for home based use. Ive gotten 90-100MB steady speeds copying to NAS on my GigE network. Streaming from NAS to 3 TV's + streaming music to two computers no problems at all. I am very pleased with this setup!

One of the 3TB Green WD's did go out, lots of checksum errors. Seamlessly replaced with another identical.

 

[cyberjock]

Did you use the wdidle tools to change the sleep time from 6 seconds to something at least 30 seconds? If you didn't you should do it. If your drive ends up with a really high value and you try to RMA it WD can and has rejected RMAs. Having a high sleep value on SMART data is an indicator you used it in a RAID setup, which the warranty doesn't cover because the drive was not designed for server use.

 

[asw2012]

Nope, I did not change the sleep times. I just RMA'd with Newegg, they did not seem to have any issues with the return.

You are suggesting to increase the idle time to at least 30 sec for performance issues?

 

[survive]

Hi asw2012,

The change isn't for performance, it just changes how long the drive waits to park it's heads. If you use the drives in an always-on type of situation and the drives are constantly parking & unparking the heads it will drive up one of the SMART counters which might make for a whiny RMA process.

-Will

 

[Reciprocity75]

"Been running this for a little over a month now, no issues with the ZFS setup. I've got 15.2TB worth of space from the 8x3TB in RAIDz2. Performance is excellent for home based use. Ive gotten 90-100MB steady speeds copying to NAS on my GigE network."

Any idea what the power draw is when idling? I am considering using an AMD trinity based system. Also, did you pick up an Intel NIC or stick with the Realtek on the board?

Thanks in advance.

Tim

 

[asw2012]

It's been at 129-131 watts idle, readings from a KILL A WATT monitor. I decided to go with the Intel NIC, did not trust the onboard. I am very pleased with the setup so far!

EDIT: as i wrote this, I was comparing my idle to my gaming rig which idles at 300+watts. Now that I've been reading up on the NAS server power @ idle, it seems as though 130 is a little high. I use a Rosewill 550w power supply, because it was just laying around. Could this be why there's a high power draw?

 

[Reciprocity75]

It does seem a bit high to me as well. For reference, I have an old Intel P45 board, Intel Q9300 core2quad, nVidia 9400GT video card and 3 old sata drives with a new 80 plus 380W power supply and it idles around 85 watts. Your A4 should be much more efficient than my Q9300 and the green drives use 6 watts or less. Your system should be under 100 watts at idle and more like in the 80 range.

If that power supply is inefficient, you will be pulling a much higher load than the system needs due to the power supply not operating in an efficient wattage. For instance, if your system needs 80 watts at idle, but your power supply is only 60% efficient at that load, the wattage at the wall will read around 130 watts. This may be what is causing the higher than expected wattage at the wall outlet. You really only need about 200watts for that A4 and the drives, but power suppllies are only getting larger and larger and the low wattage, high efficiency parts are hard(er) to find. Seasonic make some really nice lower powered units that are 80 plus bronze or better.

 

[asw2012]

I've done some futzing in the BIOS, underclocking it to 2Ghz and turned off that "turbo boost" (if I remember the verbiage correctly) now it stands idle at 105-107. I might try another PSU, try to get it down to 80ish, but how much would that really save me per year? The cost of the PSU itself?

Thanks for the reply.

 

[Reciprocity75]

I might try another PSU, try to get it down to 80ish, but how much would that really save me per year? The cost of the PSU itself?

Exactly! I am all for low wattage systems but at a certain point, the returns aren't justified by the costs. Though 30 watts saved can add up quickly over the life of the server. And the low wattage power supplies are generally less expensive, so there may be some merit to getting one.

The first thing I would do is see if you can find the specifications for your current power supply. If it is inefficient, especially at that load, it may be worth it.

 

[kosmok1000]

is ZFS raidz2 hot swap able?

 

[guldan]

I've done some futzing in the BIOS, underclocking it to 2Ghz and turned off that "turbo boost" (if I remember the verbiage correctly) now it stands idle at 105-107. I might try another PSU, try to get it down to 80ish, but how much would that really save me per year? The cost of the PSU itself?

Thanks for the reply.

Depends how much your power costs.

For me (Vancouver BC, Canada) 100w running 24/7 = $5/month (Approx)

 

[asw2012]

Depends how much your power costs.

For me (Vancouver BC, Canada) 100w running 24/7 = $5/month (Approx)

its about .155 per kilowatt-hour in Chicago IL.

is ZFS raidz2 hot swap able?

I don't think ZFS raidz2 is hot swappable. I had to "offline" a bad drive, then turn off the system and replace.

 

[kosmok1000]

Thank you!

 

[paleoN]

I don't think ZFS raidz2 is hot swappable. I had to "offline" a bad drive, then turn off the system and replace.

Not without hot-swap hardware.

 

[cyberjock]

I don't think ZFS raidz2 is hot swappable. I had to "offline" a bad drive, then turn off the system and replace.

Yeah... I don't think raidz2 means what you think it means. The RAID type, partition type, file system type, etc has zero bearing on if something is hot swapable.