Is this setup overpowered? Haswell system (E3 1230v3, C226, 16GB) - What's up with Opterons?

[Satam]

Hello,

data storage at my home is a mess and I decided I need a big storage, HDD failure proof solution. Since I like to tinker with PC hardware I decided to build a NAS myself. I guess you've heard that story a thousand times before...

So first of all here's what I need, what I do, what kind of data I have:

• My data is mostly ~80% huge media files, average file size 1-16 GB
• Then there's a 30k file music library, average file size about 20-40MB
• Apart from that it's just the usual clutter of personal files, downloaded software, documents, cat images, ...
• I don't really need high performance reads and writes, although substantially less than desktop file performance would be quite a downer
• Oh, I intend to use encryption at the NAS level, I thought about using iSCSI and encrypt on my desktop via Truecrypt, which would have the benefit that the the server hardware and home network will never see any plaintext, but I ultimately plan on using the NAS as a media server, too, and my desktop CPU has no AES(-NI) support and I really want to thin my desktop down as much as possible and make it quiet after the NAS built. So encryption has to be handeld by the NAS.

Naively following the usual recommendations like ECC, server grade hardware, a lot of RAM, relatively new hardware, I seem to have ended up with this setup:

• 2x 3TB Western Digital Red (new), 2x 3TB Western Digital Red (already bought them, now collecting dust), 2x 3TB Seagate Barracuda (already owned these for some time) - I've chosen the 3TB WD Reds because 3TB have the best value for money at the moment, I already own two 3TB drives, which limit the RAIDZ2 anyway and I really don't want to buy 6 over-priced 4TB drives at once. WD Red because IMHO the supposed extra reliablity and 24/7 capability is worth the money. 12TB or whatver I get out of them with RAIDZ2 is enough.
• Intel Xeon E3-1230 v3 (4x3.3GHz, AES-NI, HT,...) - I figured because of the HyperThreading and the fact that the price difference to the 1220 isn't that huge anyway. The 8 threads might get handy if I decide to repurpose the server after some years or make use of some of the FreeNAS extra packages. More GHz? I don't really think I need that, do I?
• Supermicro X10SLH-F - the bestprice finder site somehow narrowed it down to this board
• 2x Kingston ValueRAM Elpida DIMM 8GB, DDR3L-1600, CL11, ECC (KVR16LE11/8EF) - I'm probably going to buy another pair a few weeks later once the system is running and I have fresh cash, also they are low voltage, the mainboard supports that... good idea? difference in price is not worth mentioning
• be quiet! Straight Power E9 400W ATX 2.4 - it says it supports Haswell power saving modes, woohoo! uh, do I need that?
• Fractal Design Define R4 Black Pearl - no I didn't get the idea to buy Fractal Design from this forum, I promise... ;)

I think that's a pretty neat setup, but I have two problems with it: it costs a lot (~950€) and I guess I really don't need all that CPU power. This is my first ever hands on experience with a NAS, a self-built one even, and I have no idea what the end result would be like performance-wise. So if anyone could give me some particular advise on that, it would be great.

My guess the only thing with which I could cut down the price a bit is using a less powerful CPU with a similar board, but in the end that will not make that much of a difference because the two HDDs, the two (soon another two) RAM modules, the case and the PSU make up a huge amount of the end price, too. I thought about maybe an Atom-based CPU-soldered-to-Mainboard server, but how bad is the performance drop of an Atom C26itwas? compared to Xeon E3 12xx v3? I guess in some use cases it's quite massive, isn't it? No fun in the long run, right? Since I can imagine myself using the hardware for something else in a few years, maybe.

Then there's another minor issue I initally had with my setup: it being Haswell. Compared to Ivy Bridge there doesn't seem to be that much advantages, well, except when the CPU is doing less. Oh well the chipset is supposed to consume less power. The difference in the end price is about 60€ (Intel Xeon E3-1230 v2 + Supermicro X9SCM-F). In the end I don't really care, so I chose the newer option. But any thought on that?

Now what's up with Opterons or Athlons even?

Really, what's going on there? I tried hard to put together a system based on AMD because of cost, but it was much more difficult to do that and I gave up. It was harder to get information about the chipsets and CPUs, the options available in retail are fewer and I would probably have ended up with a desktop grade mainboard. Why is it so much harder to assemble an AMD based system? Ready made 19"-systems for a few thousand bucks, that's not a problem to get, but if you want a single CPU small office type server with AMD I somehow always ended up with "high-performance" desktop hardware. But if someone has a good enough alternative based on AMD technology, please share! To be honest I pretty much decided on Xeon now, but out of curiosity I still want to know what's going on with AMD.

Oh, then one last question: One of the Seagate hard drives I want to use for the NAS is still full, and all my backup space is already full, too (I know that's bad, but you have to understand that most of the stuff on my backup isn't really backup worthy, it just sits there because I had no other space for it to go! Hence the NAS build.) Now I have some less than 3TB harddrive which is empty laying around somewhere for sure. So my idea to get this data off that last 3TB drive in order to add it to the RAIDZ2 in the end is to create the vdev with the other empty 3TB harddrives and with an old 750GB-1TB drive as the sixth drive. Then copy the data from the not-empty 3TB Seagte to the RAIDZ2 and then replace the sixth low capacity temp drive with that same Seagate drive and thus expand the RAIDZ2 to its full capacity. Would that work out well? How long can I expect for the refurbishing(?) to take? Is there any difference between a mint 6x3TB RAIDZ2 and one that was expanded from 6x1TB to 6x3TB? I guess from all what I've heard about ZFS there absolutely shouldn't be.

 

[cyberjock]

My personal(jaded) opinion of AMD... they sell to a different target audience. Some people shop and all that chipset info, CPU design, etc is important. Other people want to see "high performance next generation computing today!" sticker on the box and buy because of that. If you read around here I'm not AMD's biggest fan. I've had problems with them and I don't buy them on purpose now. Many people here have tried AMD and had major problems that are non-descript and not-obviously pointing to a specific problem except "something is broken". Some people have no problems at all. When it comes to buying AMD I look at it with these options:

1. It may work and it is cheaper, but you are on your own if it has weird problems.
2. Buy something that is well known to work, such as one of the Intel builds.

Guess what happens more often than not? #2. And plenty of people don't want to buy AMD just so they can go buy Intel so they can do what they originally wanted.

Your plan to use the 750GB drive will work. You will need to use 8.3.1 to create the pool because 9.x has a volume manager that won't let you do anything except what it decides to let you do. And it is going to think you are a moron for wanting to use a 750GB drive with 3TB drives. Not a big deal. Just use 8.3.1 to create the pool and then upgrade to 9.2.0. 9.2.1 should have the old volume manager back in all of its glory, so you could use 9.2.1 if its released by the time you build it. It's currently in "beta" and I only recommend non-releases for testing purposes.

Resilvering will take "however long it takes". My guess is 2-5 hours. There's no short answer because there's too many factors that you have no control of.

 

[Satam]

Resilvering takes a few hours? I can definitely live with that. Well, I'll see.

I already played around with FreeNAS 9.2 in a VM and I noticed that the web interface didn't let me do as much as is advertised ZFS can do. So therefore I was initially planning to use the console for drive setup anyway. If that issue gets resolved in 9.2.1, the better. But by volume manager do you mean just the web front end or the actual zpool command?

 

[cyberjock]

When i resilvered my 2TB drive last failure it took like 20 hours.

I'd recommend against console setup. FreeNAS expects things to be a certain way and when you break those things you often don't like the results. The WebGUI has always been the problem. The CLI always works(but should be avoided). We've had so many people do CLI pools and later wonder why their pool is corrupted and unusable after a software upgrade. Do it right the first time. Trying to fix the mistakes later is impossible.

 

[Satam]

When i resilvered my 2TB drive last failure it took like 20 hours.

Hm, how full was the zpool?

We've had so many people do CLI pools and later wonder why their pool is corrupted and unusable after a software upgrade. Do it right the first time. Trying to fix the mistakes later is impossible.

Oh, that's horrible! Actually that makes me want to set up a vanilla FreeBSD properly instead. FreeNAS corrupts zpools if people set up them with the console commands directly? That's a serious showstopper.

 

[Tomas Liumparas]

FreeNAS corrupts zpools

The WebGUI setups pool with some spices. When you are dealing with the pool later, via WebGUI it expects that those spices are there. If they aren't - problems happens.

So unless you know exactly what you are doing, you should avoid it. There is a thread somewhere which describes correct procedure. But keep in mind, that you can always make some sort of mistake which you will find out about when it's too late. So try using GUI unless you really really need CLI :)

 

[cyberjock]

No, what normally happens is ZFS shows up with some error and won't mount since you didn't meet the expectations of FreeNAS. So they start doing all sorts of googling and doing stupid noob stuff. Next thing you know they've trashed the pool because for their own doing and they are asking for help. So FreeNAS is only to blame in the same fashion that a car is to blame for someone being killed by a drunk driver.

In short, you either are a FreeBSD god and you probably aren't using FreeNAS anyway because its so "limited"... -OR-

You are a noob and you shouldn't be doing crap from the CLI on any system you don't have actual serious experience with.