BUILD New Build with MB Questions

[Urlryn]

Howdy!
New to the forums and freenas but I've been skulking about reading up a lot.

I've gotten some hardware already ordered based on many recommendations and suggestions throughout the forums.

I'm looking to create a home server mainly for file storage/backups for 3 users and getting off all these USB drives i'm using now! I won't be running plex. I have a separate HTPC that would access my Blu rips from the NAS.

So for my first starting build I bought:
Case: Fractal R4
Power supply: Corsair 1050w (Though I have a fairly new Corsair 750w in my pc i could swap with)
Drives - 10x 4TB HGST Deskstar NAS drives (RaidZ2 8x2)

Now I need to purchase a motherboard....This is where i can't decide. I am already planning out a 20-24 drive setup down the road. So i just want to future proof my setup now so i don't have to worry about upgrading all the hardware later other than the case.

I've been looking at the X10SLM-F. My dealer has a refurb that he will sell me for under 100. And this includes the 8x SAS2 LSI built in. My concern with this is the memory only goes to 32GB. For my current setup..it should be fine. But down the road would this be able to handle the increase in drives?

If the above is not recommended...then my next option was the X10SRi-F. This one goes up to 512GB/256GB memory. This will need the M1015 cards.

Well depending on whats you experts recommended...will determine what memory and cpu to get! :)

Thanks
Urlryn

 

[Ericloewe]

The X10SLM-F most certainly does not have SAS ports. The one that does is the X10SL7-F.

If you want to go with Haswell-EP, there are also appropriate motherboards with controllers on board. Though, now that I think about it, those LSI SAS 3008 have been a bit problematic - separate HBAs may be safer, but more expensive. The X10SL7-F's LSI SAS 2308 works like a charm, though.

 

[Urlryn]

Ugh...thanks for the correction Eric. Not sure why I thought that one had the SAS port on it.
Yeah I saw the X10SRH-CF... but seems it would be a bit of waste to go with that at 600 (400+200 hbas).

Guess i'll start checking out CPU and Memory to go with the X10SRi-F unless someone can recommend something else.

Thanks
Urlryn

 

[Urlryn]

The Intel Xeon E5 v3 CPU's and DDR4 memory are still a pricey bunch!

Intel Xeon E5-1620 v3 Quad-Core Processor 3.5GHz 0GT/s 10MB LGA 2011-v3 - $300
DDR4 PC4-17000 • Registered • ECC • DDR4-2133 (2x16GB) - $400-500
X10SRi-F - $300
2xM1015 - $200

What are other people using with their 10-20 drive builds currently? Anything that will work and is a bit cheaper than what i'm looking at?

Urlryn

 

[hertzsae]

Since your CPU needs are not high (no plex), you could look at one of the Avaton (8 core atom based) boards. They take up to 16GB per slot, so the 4 slot boards take 64GB. I'm not sure if that's quite enough for 24x4TB drives, but it would be close.

It sounds like your needs center around mass amounts of storage and nothing else. Since you are currently looking into FreeNAS with ZFS, getting massive amounts of storage also means massive amounts of ECC memory. How important is ZFS to you? If you went with a non-ZFS route, you wouldn't have the extreme memory requirements (8GB would be fine). I'm not trying to tell you to go elsewhere, just pointing out that you may save a lot of money went with a linux or Windows server with a non-ZFS, but still redundant file system that isn't memory hungry.

Also, if you really plan to scale to 24 drives, I'd get a server case with a server PSU. That is not a normal home server build, so you should more closely be following what business users are doing.

 

[cyberjock]

No, avoton should be ignored for that much disk space. The cost of avoton + 64GB of RAM is more than the cost of a full-fledged Xeon and 64GB of RAM (plus you get the ability to upgrade to even more RAM if necessary).

This may be "for home use" but the size is far from it. Definitely going to have to look at the more expensive options that can handle more RAM. I've got 60TB of raw storage (48TB formatted with RAIDZ2) and 32GB of RAM is showing it's limits for me, even as a single user in a home environment.

 

[DKarnov]

It sounds like there are two scenarios being discussed here.

Your 10x 4TB scenario you mention first gets you a little over 25TB of usable storage. That's doable in a regular PC case like the Define R5, and 32GB is probably reasonable. You make it sound like your demands beyond raw storage are fairly undemanding. Whether you try to find a board capable of running all those drives from on-board controllers, or going with expansion cards, is up to you; the 1015s are fairly cheap and may make more sense than breaking the bank on a really fancy mobo.

20-24x is another matter. If you want to be ready for that expansion from day 1, I'd say head to eBay and find some recently modern, retired commercial server gear being resold. There's plenty of it out there. That can get you a server motherboard, CPU, and lots of memory all at once at heavily discounted prices. Alternately, get the simpler hardware for the 10x drive case now, and repurpose / resell it after you upgrade to heavier iron in the future.

Or just build another 10-drive box when you need it, and put it next to the existing one. That'll save you the considerable expense of stepping up to server-class chassis, PSU etc that you'd need for a single setup. If you had a more compute-heavy use case where the data moving around would get inefficient this might not work, but for backup and HTPC serving it seems far simpler.

 

[marbus90]

For 10 disks with Woman Acceptance Factor the ASRock E3C224D4I with the Lian-Li Q26 might be an idea. Lian Li reported that the board would fit. You could also buy more BP2SATA to hotswap them.

Indeed, above 12 drives for a media storage it gets big.

 

[Urlryn]

Hey Guys...appreciate the insight!

I just been trying to stay with in the recommended hardware....I work for a big company and have access to full Dells and HP servers including Disk Disk arrays. If i can use those then that would save some cost. But those can be very noisy and such.
Originally, as I'm able to get additional drives, I was thinking at some point I'd have 3x raidz2 (6x2). Mainly was going to have couple raidz2 for just for mass storage for my media and then a smaller one for all the family pictures and files.
So I figured i'd ask these questions and see if i'm overdoing it and should just go with something like a 8x2 setup instead.

Would it be better to just increase drive sizes or to add more drives?

Urlryn

 

[marbus90]

Upgrading to 6TB disks may still save you money compared to the additional chassis needed for further 10x 4TB drives, altough 7200rpm 6TB ones aren't exactly cheap.

For 24 bays you could use a Norco Rackmount chassis, throw 3x120mm (less recommended for 7200rpm disks, since there could be hotspots) or 10x80mm (2 rows of 5x80mm, less hotspots) fans in, use 2x M1015/H220 cards on a Xeon E5 mobo. Base cost for 10 drives would be comparable for X10SL7 with a Xeon E3 versus a 2011-3 board with a low-end Xeon E5, which already has 10x SATA onboard. RAM doesn't need to be much more expensive if you plan on 32GB - Registered 16GB DIMMs may be cheaper per GB than 8GB unregistered.

Supermicro chassis would be comparable to the Dell/HP rackmount servers in terms of noise.

 

[Urlryn]

After sitting down and thinking about what I was attempting, especially after reading everybody's comments.
For now I'm going to forget going to such a large setup with the 24disks. I think will stay with my current plan with just the 10disks. If i keep it cleaned up a bit it shouldn't ever got maxed...at least not for a while!

On a side note...Now DKarnov mentioned using commercial server hardware. Are there any particular that can be used? I have access to Dell and HP servers easily enough such as PowerEdge 2950, 6850's, DL360 and DL380's.

Now that being said....I'm still looking at making sure i have enough memory. I should have 40GB at a minimum.
If so...it looks like my only options are still the X10SRi-F, X10SRL-F, X9SRL-F (about $200-300) and the X9SRH-7F ($400 / Ebay had one for 288). Out of these the X10SRi-F is my best option price wise for the board itself.
But for memory and CPU...are the ones listed above the best (pricing and performace) i'm going to get?

Thanks for everybody help!
Urlryn

 

[Ericloewe]

Well, DDR4 stuff is still rather expensive. Deals on Ivy Bridge-EP and DDR3 should be easier to find, though.

As for pre-built servers, just make sure FreeNAS installs and boots correctly on them before committing. Many have weird issues that are hard or impossible to solve. Typically they're stable after they've booted, other than nasty lock-in techniques like proprietary firmware on SAS controllers.

 

[marbus90]

It depends on the generation of the servers. a DL180/380 G6 upwards with 12 hotswap bays could be a nice choice, I'm not very into Dell servers to give a from-memory advice there. Also you could go 2x 6disk z2 which is pretty much the standard choice. Just keep in mind the general 1GB RAM per 1TB Storage rule and you're fine.
Personally I'd pick a Server with at least E56xx CPUs for AES support, E55xx CPUs may still be enough for your use case.

Also for sequential workloads (which BR-rips are) 5400rpm drives are the better option. You can probably get 12 5400rpm drives and more performance out of the two raidz2 compared to a single big z2 of 7200rpm drives.

 

[Urlryn]

Sorry Marbus...i should be been a little more clear in my initial post. I fixed that.

I already have the Case/Power Supply/Hard Drives. I only got the 10 drives since the case only supported that many.

I assumed that since this box wasn't going to be doing anything like the Plex transcoding and/or ripping Blu's from this system (external htpc box does that), I could get a bit more space using the 8x2 z2 setup.

This is just planning and learning phase atm. So if i need to return something and get something else I can still!