BUILD Home / Office Build Critique

[Mguilicutty]

Hello everybody, been lurking, first post, and I apologize in advance for it's length...

I've been planning for a while to merge all my various servers into a virtualized setup and using Freenas as the storage end of it. I'm new to Freenas but I've been reading up on it for a few months and I've been running it under Virtualbox for a while trying out different things and familiarizing myself with it. My home is my office. I'm a network guy during the day, but I also have a couple of side businesses doing general IT work for a few small businesses and a data processing deal. I also have all the modern home storage needs like videos, photos etc. I've been in IT for 20+ years starting out in sys administration and coding but the last 8 years or so I've concentrated mostly on the network side of things. I have no real experience with NAS admin, I've always used raid cards on my servers.

My thoughts have been elsewhere for a while and I'm ending up having to put this together sooner than I had wanted to. I was hoping to bounce it off you guys and see what your thoughts were. I'm building a new box for the Freenas machine, I've purchased everything except for the ram and SSDs. Here are the specs so far...

Supermicro 933T case (3U 15 3.5" SAS/SATA)
Supermicro X9DR7-LN4F Motherboard (4x Intel Gb, 10 SATA via Intel, 8 SAS/SATA via LSI 2308)
Intel E5-2620 cpu (6 cores / 12 threads 2GHz, 2.5 turbo)
64 GB ECC RAM (4x16GB)
6x WD RED 3TB SATA in raidz2 for 12TB slow pool
4x WD RE 2TB SAS in raid 10 for 4TB fast pool
4x ? 240GB SATA SSD in raid 10 for 480GB real fast pool

Planned storage use...

Slow pool

Family photos / videos - presently ~900GB and growing all the time.
Music - only ~30GB now but I'm going to set my wife to ripping all our old CDs + tapes + vinyl into a lossless format.
DVDs / BluRays - approaching 3TB and getting nuttier w/ BluRays.
Rsync - now ~2TB. I do offsite backups for the small businesses I do IT stuff for. Growing slowly.
General storage - ~1TB + for a bunch of junk.
DP scratch space - was hoping to have around 6TB of temp storage space for my DP work.

Fast pool

Time Machine - 2TB or so
Wifey space - Wife needs ever more IPhoto space... 1TB+ ISCSI or AFP...
Raspberry Pis - iSCSI boot and storage for a couple Pi's around the house.
Some low use small databases etc...

Real fast pool

iSCSI boot and storage for VMs.
Exchange, SQL and MySQL DBs.

The motherboard has 4 Intel Gb ports on it and I was going to LAG 2 each to data and iSCSI vlans at first, maybe go 1 data and 3 iSCSI if necessary. I have dedicated boxes for a firewall and the DP machine and the VMs will be hosted on a Dell C6100 that is on it's way and are a mixture of 08 and 12 Servers as well as a few Linux and BSD boxes. I'm still torn between VMware and HyperV, I might end up using both. Hopefully VMware announces a relaxation on their 32GB policy next week...

I have a question regarding expansion of the slow pool. If I end up down the road needing more space is my best option to basically duplicate that (6x3TB z2) in a second chassis off of another LSI / IBM card? I can deal with that. Second would be the sustained write speed on that pool, I'll find out down the road but can anyone guesstimate what I should be expecting? My data sets are typically 2 to 3 TB (they arrive on tape) and I would like to store them on this pool (not work them on the pool, just temp storage not on tape).

So, am I asking for trouble? Anything I haven't thought of? Anything I should look out for?

Thanks,
Chris.

 

[Mguilicutty]

Nobody loves me?:( lol

 

[cyberjock]

We hate you!

Actually, your build is quite complex. And speaking from watching other newbies, you are very likely to have an uphill battle setting up iSCSI with ZFS. Lots of forum posts on the topic if you want to read more about it.

In terms of expectations there's no way to really guess what the best choice is that far down the line. You should plan your pools to last you at least 18 months because doing upgrades isn't all that trivial, is expensive to "fix" an improper build, etc.

But your ideas seem fairly sound. Your "fast" pool is probably going to require ZFS tuning, and a ZIL.