Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

Guide: How much will a proper home FreeNAS setup cost me?

Status
Not open for further replies.

anodos

Belly-button Lint Extraordinaire
iXsystems
Joined
Mar 6, 2014
Messages
5,416
Thanks
1,488
#41
its possible to put the system part of this together for a few hundred dollars even using new gear. ULTRA low cost could use
Gigabyte GA-990fxa-ud3
AMD cpu of your choice ranging from 80$ to 200$ depending on needs
dell perc h310 or h200
intel pcie 1gb nic
nzxt source 210 case 'holds more than 8 3.5" drives'
you will need a cheap video card.
and good power supply.
ECC NON REG RAM (this stuff is kinda pricey but still a lower cost overall build)
this is around 500$ (depending on cpu and amount of ram, you want atleast 8gb, 12gb or 16gb is better)


but even going with the AMD G34 server route and using a supermicro board you can get a complete system with enough horse power to be a VM host and still come in well under 1000$
IE: swap the above MB with an H8SGL, G34 cpu, and ECC REG ram (the ram is dirt cheap, get like 32gb or more) and suddenly you have a beast of a rig. this stuff is still in production today even though it is a few generations old, so its cost has been driven way down but you will not over utilize it even in a server environment.


when money is freed up i personally prefer the Chenbro cases, then supermicro, and only if i found one for nearly free would i use a norco case again. however given the choice i would probably assemble a nzxt source 210 (this case will actually fit sideways in a rack, use a rack shelf or get creative with a universal rail kit) over some of the low end norco cases.

you do not have to burn money to use freenas contrary to some of the 'wisdom' on this board.
Excellent! I haven't seen any "AMD vs Intel" (regarding ECC, FreeBSD compatibility) forum discussions for a while. For those too lazy to wait for this to build up steam, please refer to previous forum threads in this vein. Price differences are way overstated.

For the sake of brevity, consider the following points:
  • Generic pro-Intel arguments 1, 13, 42
  • Generic anti-AMD counter-arguments 2,19,59

I think those pretty much settle it.
 

mattlach

FreeNAS Experienced
Joined
Oct 14, 2012
Messages
280
Thanks
8
#42
Excellent, I haven't seen any "AMD vs Intel" (regarding ECC, FreeBSD compatibility) forum discussions for a while. For those too lazy to wait for this to build up steam, please refer to previous forum threads in this vein. Price differences are way overstated.
Again, read my previous post. ECC support is far from guaranteed, and difficult to confirm that it is working.
 
Joined
Aug 1, 2014
Messages
37
Thanks
4
#43
i am currently running a UD5 with ECC. there is some set up that MUST be configured in the bios to enable ECC function. simply dropping in ECC does not enable ECC like a server board would. also there are some linux utilities to display rather the kernel shows ECC in use. after configuring the bios i used a live CD to 'verify' that the world thinks the system actually is ECC aware. it reports that it is, in the same way that my G34 system reports that is ECC aware. so i have to theorize that it is in fact using ECC function. (nearly all AMD CPU's do actually allow ECC, its up to the MB to allow activation)

its not plug and play, but no ubber low cost yet functional system is. like cross flashing an ibm hba to lsi IT mode. think of it as a hacker/tweaker build. if your o.k. with spending time to save money, then its a good reliable configuration.

the gigabyte boards all show SEVERAL sections for ECC in bios when ECC ram is installed, the ticket is finding all of them and setting them to valid ECC ENABLED status. it is a process, it is not difficult.
 

mattlach

FreeNAS Experienced
Joined
Oct 14, 2012
Messages
280
Thanks
8
#44
i am currently running a UD5 with ECC. there is some set up that MUST be configured in the bios to enable ECC function. simply dropping in ECC does not enable ECC like a server board would. also there are some linux utilities to display rather the kernel shows ECC in use. after configuring the bios i used a live CD to 'verify' that the world thinks the system actually is ECC aware. it reports that it is, in the same way that my G34 system reports that is ECC aware. so i have to theorize that it is in fact using ECC function. (nearly all AMD CPU's do actually allow ECC, its up to the MB to allow activation)

its not plug and play, but no ubber low cost yet functional system is. like cross flashing an ibm hba to lsi IT mode. think of it as a hacker/tweaker build. if your o.k. with spending time to save money, then its a good reliable configuration.

the gigabyte boards all show SEVERAL sections for ECC in bios when ECC ram is installed, the ticket is finding all of them and setting them to valid ECC ENABLED status. it is a process, it is not difficult.
It might be working then, though I'd still not feel entirely comfortable.

Don't rely on the programs showing ECC, linux or ortherwise. My reading suggests that they just report that ECC ram is installed, not that it is infact functioning. Even Memtest86+ isn't guaranteed to tell you if ECC is actually working.

It would be fantastic if there were a tool that could intentionally write single bit errors in RAM, and see if they got corrected, but last time I read up on this there was no such thing.
 
Joined
Aug 1, 2014
Messages
37
Thanks
4
#45
the linux kernel can tell the difference between ECC ram installed, and ECC ram in use. heck there is a module that you can load to force use ECC in a linux kernel even if the bios only reports it as 'installed' but does not allow it to be 'enabled'. (not helpfull on freenas, but still allows you to see the functionality of the system)

you are correct in the fact that there is no real utility to TEST this. but i am o.k. with accepting that the technology in place tells me it is working.

if i get a chance i will post a log showing the report, it shows several places and all must say enabled or active. i tried grabbing the log but my web based terminal plugin doesnt scroll or output to a file. i will try grabbing it when i am onsite this weekend.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,025
Thanks
3,880
#46
It would be fantastic if there were a tool that could intentionally write single bit errors in RAM, and see if they got corrected, but last time I read up on this there was no such thing.
But such a thing is possible on Intel processors. I looked into it some months ago and Haswell's documentation includes ECC error injection, controlled by a few registers. Curiously, I couldn't find their equivalents on the Haswell-EP documentation.
 

David Dyer-Bennet

FreeNAS Experienced
Joined
Jul 13, 2013
Messages
223
Thanks
14
#47
Glad to hear Intel has ECC options in the non-Xeon range now! For years people were pretty much stuck with AMD (which was mostly a pretty good deal for this sort of server, but still, choice is good).
 

nightshade00013

FreeNAS Expert
Joined
Apr 9, 2015
Messages
1,259
Thanks
293
#48
For what it is worth, by cruising eBay you can get some tremendous deals on server pulls.

As long as you run stability tests / RAM tests prior to setting everything up, reliability is not an issue, and they save you SO much money.

I built mine out of a unused old stock Supermicro dual socket westmere era motherboard, 96GB of used registered ECC RAM, and two low power 6 core Xeon's.

In the end I spent what one would have for consumer grade hardware, but got something much much more.

My build looked something like this, but it was over time, so I salvaged parts here and there, didn't buy them all at once. This was about a year ago, so probably cheaper now:

CPU: Pair of L5640 Xeon CPU's: $120 for the matched pair used on eBay
Motherboard: Supermicro X8DTE: $150 New on ebay. No I/O plate
I/O Plate: $8 on eBay
RAM: 12 8GB Hynix low voltage DDR3 Registered ECC sticks. $40 per stick from seller in Hardforums FS/FT section
SAS Controllers: 2x IBM M1015 for ~$100 each on eBay
Case: Norco RPC-4216 $320 on Amazon
SAS Cables: 4x for $39 on Monoprice
PSU: Antec Earthwatts 550W 80 plus Platinum: $95 on Newegg
Adapter cable for second EPS power: (PCIe to EPS): $6
CPU Coolers: 2x SNK-P0040AP4: $48 for both on Amazon

Sub total $1,466

So, as you can see, the only place I really splurged was on the 16 bay Norco server case, everything else I got pretty fantastic deals on.

Of course, I spent almost $3000 on drives (12x WD RED's) on top of the above, but you have to spend on drives either way. Can't get away from that if you need storage.
This is extremely close to the builds I am working up. I have one with the X8DTE that my father has in CA mounted in a 1U server case that will likely end up in the Philippines in a year or two with a pair of E5606's. He has to pick up the ram and drives but I was able to pick up the case, psu, mb and cpu's for 275 plus shipping from ebay. It did come with 24GB of ram but I kept that for myself.

I am using a X8DT6-F (I have a second one that the video does not work on but functions with a video card inserted) with a pair of E5640's and 6 4gb sticks of ecc ram that came from the other server. I am going to be using Plex quite a bit since I don't have cable so that everyone in the house can watch whatever they want. Mumble server (btw thanks DrKK for the tutorial). And I will have a webserver set up in a jail for basic goofing around. VPN will also be setup and some other stuff working in the background. I did go a bit high on the PSU at 750 watt but it came with support for dual EPS for the board and I will be able to power all the drives when it is running full bore. Calculations show it's actual use around 450 watts under 90TDP load but I don't know exactly how hard it's going to work but even at 50%TDP it will be in the 400 watt range.

Right now without drives and a few other things I have approx 350 into my build. If I didn't want to have more than 4 drives I would have been able to do it for the 275 plus drives since that was pretty much a complete package. Power use will be a little higher on older hardware but I currently have two systems running that will go offline that this one system will replace so I am not all that worried.
 

Xelas

FreeNAS Aware
Joined
Sep 10, 2013
Messages
91
Thanks
14
#49
I have a X10SL-F (fantastic board, and having IPMI is great!), and I got a Xeon E3-1245 because I splurged. I LOVE the hardware, drives, etc. - it's been a rock-solid system now for 2 years.
I also got a Lian-Li PC-V354 case, which I am growing to HATE. It is great to set up, it's compact, and I fit 7 full-sized drives and 3 SSDs into it without breaking a sweat, but the sides attach via a bunch of little screws, and the fan filters don't come out without major endoscopic surgury, and getting them out is iffy enough that I shut the server down in case I pop a cable free.

When getting a case, look into one that is easy to maintain and service!
 

diskdiddler

FreeNAS Guru
Joined
Jul 9, 2014
Messages
2,133
Thanks
126
#50
Is there any progress on MiniITX boards with
10 sata ports
4xUSB3
ECC DDR4 support
REASONABLY priced, low / mid power CPU's (sub $350 at most) yet at LEAST 3 to 4x performance of AMD Turion N54L?
Very likely FreeNAS10 support

I'd love to be able to use an i5 6400 with 32GB of ECC somehow at a 'semi' reasonable price :/
EDIT: a low grade Xeon is fine by me, but I'd really like MiniITX and the cost of board, CPU, RAM to be less than $650 US
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Thanks
1,090
#52
i5 does not support ECC.
Indeed. I am actually quite surprised, diskdiddler has been with us a long time, and I can't imagine not knowing that i5's and i7's don't support ECC; it's probably #4 or #5 on the list (not yet made, but we should probably make it) of "Things FreeNAS People Know".

As for your price target, and the desired specs, I think you should add a "pony" also, in your list of requirements, sir.
 
Joined
Dec 1, 2015
Messages
55
Thanks
5
#54
Is there any progress on MiniITX boards with
10 sata ports
4xUSB3
ECC DDR4 support
REASONABLY priced, low / mid power CPU's (sub $350 at most) yet at LEAST 3 to 4x performance of AMD Turion N54L?
Very likely FreeNAS10 support

I'd love to be able to use an i5 6400 with 32GB of ECC somehow at a 'semi' reasonable price :/
EDIT: a low grade Xeon is fine by me, but I'd really like MiniITX and the cost of board, CPU, RAM to be less than $650 US
That's a tall order. How about this ASRock C236 WSI? ($250)

http://www.asrockrack.com/general/productdetail.asp?Model=C236 WSI#Specifications

It has 8 SATA3 ports, but otherwise fits the bill with:
- 4 USB3
- ECC DDR4 up to 32GB (2x16GB)
- 2 Intel NICs

Paired with E3-1220 v5 ($199) and Crucial CT2K16G4RFD4213 (2 x 16GB DDR4 ECC, $232) comes to $679.

CPU Passmark score 7613 against 1399 for the N54L.

Sent from my Nexus 6P using Tapatalk
 

mattlach

FreeNAS Experienced
Joined
Oct 14, 2012
Messages
280
Thanks
8
#55
Is there any progress on MiniITX boards with
10 sata ports
4xUSB3
ECC DDR4 support
REASONABLY priced, low / mid power CPU's (sub $350 at most) yet at LEAST 3 to 4x performance of AMD Turion N54L?
Very likely FreeNAS10 support

I'd love to be able to use an i5 6400 with 32GB of ECC somehow at a 'semi' reasonable price :/
EDIT: a low grade Xeon is fine by me, but I'd really like MiniITX and the cost of board, CPU, RAM to be less than $650 US
I'm curious. What about MiniITX is so attractive to you?

I would have thought that any case large enough to fir the 10 drives you are looking to fit (based on your SATA requirements) would be at least large enough to fit a Micro ATX or full size ATX, and maybe even an EATX board.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Thanks
1,090
#56
I'm curious. What about MiniITX is so attractive to you?

I would have thought that any case large enough to fir the 10 drives you are looking to fit (based on your SATA requirements) would be at least large enough to fit a Micro ATX or full size ATX, and maybe even an EATX board.
I personally don't give a crap about mini-ITX, but I think I have some ideas why others do. There's a couple of answers to this.

"What about an ITX is attractive to you?" Meh. What about Angelina Jolie is attractive to you? I never understood it. Some people like what they like, and there's no point in asking. I think 85-90% of the people simply "want" it, and can offer no more cogent explanation.

For the remaining 10-15%:

There are a number of super sweet cases built for mini-ITX *and* a high drive count. i.e., built for Free-NAS. I will admit, some of the mini-itx offerings from, say, Fractal Design, are really elegant.

Also, some of us are married to spouses who think we have "too much computer crap" and set Nazi-like restrictions on how much additional footprint will be tolerated.

There appears to be, these days, a bit of pimp prestige (for some reason) is having a lot of computer crap crammed into a fancy, tiny case. People will gladly spend hundreds of dollars more to get special cooling solutions (because the case is so small and standard cooling won't work), then post ceaselessly, pictures on imgur with their new builds: "Hey guys, I have 1500W of TDP between the CPU and the GPU in this case that is smaller than Jordan Hubbard's right ass cheek!!! Next step, is to passively cool the entire thing so that it is ***silent***".

Anyway, people have their reasons, and, as I said, there are a number of admittedly elegant cases that will hold 6 or 8 or 10 drives that are no bigger than a toaster that require mini-ITX motherboard.
 

mattlach

FreeNAS Experienced
Joined
Oct 14, 2012
Messages
280
Thanks
8
#57
I personally don't give a crap about mini-ITX, but I think I have some ideas why others do. There's a couple of answers to this.

"What about an ITX is attractive to you?" Meh. What about Angelina Jolie is attractive to you? I never understood it. Some people like what they like, and there's no point in asking. I think 85-90% of the people simply "want" it, and can offer no more cogent explanation.

For the remaining 10-15%:

There are a number of super sweet cases built for mini-ITX *and* a high drive count. i.e., built for Free-NAS. I will admit, some of the mini-itx offerings from, say, Fractal Design, are really elegant.

Also, some of us are married to spouses who think we have "too much computer crap" and set Nazi-like restrictions on how much additional footprint will be tolerated.

There appears to be, these days, a bit of pimp prestige (for some reason) is having a lot of computer crap crammed into a fancy, tiny case. People will gladly spend hundreds of dollars more to get special cooling solutions (because the case is so small and standard cooling won't work), then post ceaselessly, pictures on imgur with their new builds: "Hey guys, I have 1500W of TDP between the CPU and the GPU in this case that is smaller than Jordan Hubbard's right ass cheek!!! Next step, is to passively cool the entire thing so that it is ***silent***".

Anyway, people have their reasons, and, as I said, there are a number of admittedly elegant cases that will hold 6 or 8 or 10 drives that are no bigger than a toaster that require mini-ITX motherboard.
Don't get me wrong.

I understand the desire for Mini ITX desktops. I did the SFF thing for my desktop back in 2010 with a Core i7-920 system in a Shuttle barebones SFF case. I changed my tune after a while when I constantly had to fight the thing for expansion space, and more power than the little pygme PSU could provide. The appeal was for its nice small sleek looks, and taking up less space. In reality - however - i found it used MORE of my usable space, since they take desk space, and my towers always sit on the floor, and thus take up no desk space at all.

Either way, long story short, I got over my SFF phase because it took more usable space, limited my expansion and place the fans closer to my ears, making noise a bigger issue. That and it was more expensive than building something equivalent but larger. Now my main system is in a Full Tower, but I'm kind of a power user.

It didn't make any sense to me from a storage server perspective though.

Firstly, I didn't realize there were Mini-ITX based high storage capable cases. My assumption was putting that many drives in one would mean it needed to be large anyway, so why not just go with a big board. How do they do it? Do they use 2.5" laptop drives?

My server is big, and makes a decent amount of noise, but it doesn't matter, as it is in the basement, out of earshot, where there is plenty of space.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,025
Thanks
3,880
#58
How do they do it?
Like this. That thing even takes ASRock's voodoo-shrunk X10SL7-F with the absurdly complicated model number.

Disadvantages:
  • Cooling is harmed by the desire to have a clean front panel
  • The backplanes they sell for this thing (they only include one for two drives) don't support hot swap
  • ASrock's board is the only popular board that really makes this chassis really useful
  • It is a bit cramped
  • It's expensive as hell, due to it being made of aluminum.
 

diskdiddler

FreeNAS Guru
Joined
Jul 9, 2014
Messages
2,133
Thanks
126
#59
I'm curious. What about MiniITX is so attractive to you?

I would have thought that any case large enough to fir the 10 drives you are looking to fit (based on your SATA requirements) would be at least large enough to fit a Micro ATX or full size ATX, and maybe even an EATX board.
I am working with a ventilated but confined space. mATX is the largest I would even consider, prefer mITX.
I'm currently using a HP Microserver N54L - it's a proprietary board design (to my knowledge) about half way between mATX and mITX.
I've seen several mITX boards capable of some half decent stuff but not a lot I admit.

If you use decent quality fans, decent quality enclosure and all well designed. A 6 disk mITX is viable (The Node 304 case comes to mind)
(I may want 8 to 10 sata ports, but I don't intend to exceed 6 physical hard disks ever)
 

diskdiddler

FreeNAS Guru
Joined
Jul 9, 2014
Messages
2,133
Thanks
126
#60
NOTE: I'm also one of those tiny case guys. I'm sick of big, reverberating noise boxes, taking a big footprint, making echo / reverb noise, being unable to fit into certain locations.
My HTPC is a very very small case, yet capable of a full length gaming card (even tho I've not got one in there)

I suspect my next FreeNAS machine to go in a Node 304 case TBH, unless something else comes along.
Also this all depends on SSD's, if there's some miracle and 4TB SSD's drop to sub $200 US in the next 12 months, then I may look at a case which could hold say 10 x2.5" instead of 6x3.5" (although SSD's would be quieter / cooler anyhow)
 
Status
Not open for further replies.
Top