Looking for advice to get it right.

[Lucien]

I'd like to get some advice on building a new FreeNAS unit. My objectives is to be frugal with my choice of components, but at the same time do it right. (I'm a big believer in aiming to get it right the first time around.) This box will be storage only, so it should be undemanding in the CPU department.

A note on suppliers: The easiest choices for me are Amazon or local shops. So I've included Amazon links where relevant, but not for everything.

On X10 vs X11 based Supermicro platforms: A quick look at suppliers seems to indicate that X11 motherboards can be had for cheaper than X10 ones. On the other hand, I'm under the impression DDR4 RAM is at at high, which might more than make up for the difference. I'm also not sure if an X10 based platform is still a viable build because of CPU availibility, but here's my list so far:


X10 System:

Motherboard - Supermicro X10SL7-F (about $240)
Newegg
Includes SAS2308, but I could just get a M1015 if I go with a different board that doesn't have it.

OR

Supermicro X10SLL+-F-O (about $185)
Amazon
Not the X10SLL-F. According to the Supermicro web site, this doesn't have the Intel i217LM. Does that matter?

CPU - G3260 from a local retailer. (About $100)
Actually costs more than the X11 based CPU options, and I'm not sure if it's still in stock. I also don't know of any reliable source for alternative options.

RAM - Crucial CT2K102464BD160B ($137)
Amazon
I think, but am not completely sure if this is compatible with the X10 motherboards.


X11 system:

Motherboard - Supermicro X11SSM-F
Amazon
The price on this has been jumping around quite a bit, and I've seen it on par (or cheaper) than the X11SSL-F. Not to mention Amazon currently only lists third party sellers for the X11SSL.

CPU - G4400 ($54 Amazon or $65 local) or G4560 ($70 Amazon or $71 local)
Is there any point in going Kaby Lake over Skylake aside from a 200 MHz clock bump? Also, the sellers on Amazon are currently third parties. The hardware guide offers the G4600 as the Kaby Lake option.

RAM - ???
This is where I'm lost. Crucial's website lists options, but they're all out of stock, and I don't know what the matching part number is for an Amazon search. I'm not sure if Kingston RAM is still to be avoided.


Common parts:

Boot drive - 2x Samsung 850 EVO 250 GB ($85 each)
Amazon
For security I'd like to go wtih the mirrored boot drive option. But are dual SSDs overkill? My other option is dual USB flash drives, but that doesn't seem to be advised. If not I'd have gone with dual Sandisk Fit drives.

Storage - 6x HGST NAS 4TB drives
Probably the easiest part to pick. Configured as a RAIDZ2.

PSU - Seasonic Focus Plus Gold 650 W
Seasonic seems reliable, and from what I've read here 650 W seems appropriate for the box, with minor room for expansion. Would 550 W do though?


Any comments or suggestions on my component choices? In particular, I could really use advice on what DDR4 options are available.

 

[Inxsible]

As you have probably noticed, DDR4 is expensive at the moment due to shortfall. If you choose to install the least amount of RAM required for now, then you can go with the DDR4 option. You can then upgrade the RAM at a later point when prices have stabilized a bit.

I have used a single USB for my boot drive. This was driven partly due to the fact that the board I used didn't have enough SATA ports and also due to the fact that my system is not mission critical. If my machine is down for a day, the only thing that would happen is that my kid would bawl her eyes out since she cant get to see Frozen or Moana. I use Youtube Streaming at that point. If you have spare SATA ports or if you need to have a true 24/7 machine then it's best to go with a SSD as boot. Mirror if you can spare an additional SATA port and some more money. See if you can find a smaller SSD. 250 GB seems overkill for the amount of OS data that is installed. See if you can find a 16 or 32GB SSD

For PSU, I would go with a 650 - which would allow a bit of room

 

[geekmaster64]

As you have probably noticed, DDR4 is expensive at the moment due to shortfall. If you choose to install the least amount of RAM required for now, then you can go with the DDR4 option. You can then upgrade the RAM at a later point when prices have stabilized a bit.

I have used a single USB for my boot drive. This was driven partly due to the fact that the board I used didn't have enough SATA ports and also due to the fact that my system is not mission critical. If my machine is down for a day, the only thing that would happen is that my kid would bawl her eyes out since she can't get to see Frozen or Moana. I use Youtube Streaming at that point. If you have spare SATA ports or if you need to have a true 24/7 machine then it's best to go with a SSD as boot. Mirror if you can spare an additional SATA port and some more money. See if you can find a smaller SSD. 250 GB seems overkill for the amount of OS data that is installed. See if you can find a 16 or 32GB SSD

For PSU, I would go with a 650 - which would allow a bit of room

+1 for that advice.

 

[Inxsible]

Oh! just noticed that i double posted.

EDIT: now deleted the double post.

 

[geekmaster64]

Oh! just noticed that i double posted.

EDIT: now deleted the double post.

That'll happen! :P

 

[Maelos]

Take a look at my build thread (in signature), we had a lot of the same questions. It appears that your hardware is good for what you are going for, but I want to stress that gathering your requirements is the most important step in the process. Every heard of the 7 Ps? I don't mean to say you have not already done this, but I always think it is worth a recheck on what you want the machine for now and what you may want it for in a few years. With that being said, here is my interpretation per part. Take this with a grain of salt as I don't really have any experience with FreeNAS operation itself as I am in the midst of the build and configuration process myself.

X10 System:
Motherboard - Supermicro X10SL7-F (about $240)
Includes SAS2308, but I could just get a M1015 if I go with a different board that doesn't have it.

While having an included SAS controller is nice, often the deals that I found on eBay were cheaper than buying the parts peicemeal. This is just food for thought and
Actually costs more than the X11 based CPU options, and I'm not sure if it's still in stock. I also don't know of any reliable source for alternative options. You have to decide whether you want to save some cash or have absolute control over the parts, or somewhere in beween.

CPU - G3260 from a local retailer. (About $100)

CPU is not going to be your issue unless you run lots of VMs/Jails or have heavy traffic. CPUs can be upgraded also for fairly cheap.

RAM - Crucial CT2K102464BD160B ($137)

DDR4 is ludicrous expensive right now. Anythind DDR3 I approve of. Registered seems to be cheaper than Unregistered/Unbuffered right now, so take that into account.

X11 system:

My advise is to just say no. The RAM is double the cost of DDR3 ram of the same type/size.

Boot drive - 2x Samsung 850 EVO 250 GB ($85 each)
For security I'd like to go wtih the mirrored boot drive option. But are dual SSDs overkill? My other option is dual USB flash drives, but that doesn't seem to be advised. If not I'd have gone with dual Sandisk Fit drives.

I think 250 GB is overkill for FreeNAS, but someone please correct me if not. I think 64GB is the recommended minimum. Again this depends on how many VMs/Jails you plan to run inside of it. Id rather you put the money into drives.

Storage - 6x HGST NAS 4TB drives
Probably the easiest part to pick. Configured as a RAIDZ2.

Yes. I have yet to use, but am excited for my WD NAS drives (WD owns HGST)

PSU - Seasonic Focus Plus Gold 650 W
Seasonic seems reliable, and from what I've read here 650 W seems appropriate for the box, with minor room for expansion. Would 550 W do though?

There are a couple calculators out there but honestly the best way, I think, is to measure it live. Do you have a similar device at home? I'd advise purchasing a Kill-a-Watt or something like a HS110 from TPLink https://www.amazon.com/Smart-Wi-Fi-Energy-Monitoring-TP-Link/dp/B0178IC5ZY will let you remotely power on/off and will monitor electricity usage (got the idea from crypto mining research). For peak efficiency, take your usage wattage number and double it. Shoot for for using 50-60% of your power supplies's max capacity and never go above 80% as it can cause degradation of the PSU. If you can spare a few bucks, or have saved money elsewhere in the build, I'd try to go for a higher efficiency standard and or modular cabling to make the build cleaner/easier. I am by no means an expert on PSUs, but those are some of the highlights I have gathered.

 

[tvsjr]

Dual SSDs for boot are high on the paranoia level. If you're talking mission-critical business, absolutely... for a home box, meh. Regenerating the system is easy if you back it up routinely (to a drive that doesn't live on the FreeNAS).

The boot drive size is 16GB or better. http://www.freenas.org/hardware-requirements/
I use a pair of Intel 320 40GB SSDs that I bought off eBay for $20/ea.

 

[joeinaz]

When looking at your build one key question comes to mind; how much storage do you plan to start with and how much do you plan to grow. There are a number of threads talking about cost optimization. I have three SuperMicro boards X8, X9 and X10; here is a summary of the differences between the 3 boards:

Memory: The key cost factor today is ECC memory. DDR4 cost kept me away from newer solutions. My X8 and X9 board use DDR3 RDIMMs. I recently picked up 24GB for under $100. My X8 board has 12 DIMM slots and supports up to 192GB of RAM. My X9 has 8 slots but supports up to 512GB of RAM. My X10 board has only 4 DIMM slots. It uses expensive UDIMMs and is limited to 32GB of RAM.

CPU: My X8 board offers the lowest cost in CPU of the three solutions. A single E5640 offers similar total performance as the G4560 you are considering; the big difference if the E5640 goes for about $10 and I can put 2 of them on my motherboard. My X8 is using an E5-2650; this processor has about twice the aggregate performance of the G4560 and cost about $70. My X10 is running a G3258.

PCIe Slots: Sometimes even the number and type of PCIe slots can be a factor. My X8 and X9 boards have plenty of x4 or larger slots but X10 only has 2. This was a hindrance as I was building and all in one test system that includes an M1015 SAS card , a SAS expander card and a card for a tape drive (3 slots needed)

The bottom line is right now, IMHO, the X9 board seems to be the sweet spot for price/performance for FreeNAS builds.

 

[Lucien]

Thanks for all the comments and advice. Sorry for being slow in getting back to this, it's been a busy two weeks...

Ram: Yes, I'm aware DDR4 prices are mad at the moment. I was planning to go with the minimum for now, and hopefully pop in an extra stick (pair?) when prices come back down.

I know the rule is 1GB per TB of storage, but the hardware guide also says there's a 8GB minimum. Could I get away with 8 GB for a 6x 4TB raidz2 setup, or is that asking for trouble?

Also after searching around, these appear to be the part numbers for compatible Kingston ECC units:

KVR24E17D8/16
KVR21E15D8/16
KVR24E17S8/8
KVR21E15D8/8

Are those a safe choice? I've seen some of those mentioned on these forums for proposed builds, but not any user experiences.

X10 vs. X11: Totaling up the cost for motherboard, CPU and RAM, they actually come out about equal. The X11SSL-F is currently going for $150 and the CPUs are cheaper than their X10 counterparts (for me at least). That ends up offsetting the RAM cost. Not ideal, but if I go with the minimum RAM for now...

Shopping for older second hand components is an option I'd rather not take, as there's extra work hunting them down, the uncertainty about reliability and warranty, etc. It may be the only way to go for a small SSD though...

Boot drive: I'm not running mission critical, so I would be willing to consider dual USB flash drives. What would be the worst that can happen if they were to fail? Assuming I kept a copy of the config in a safe place.

Alternatively I may have a ~120 GB SSD in a system I'm not really using anymore and could repurpose that as one of my boot drives. Is it possible to mix a 16 GB flash drive with a 120 GB SSD?

What should I look out for if buying a second hand SSD (on eBay or similar)?

PSU: My choice of the 650 W unit was based on an article I think I read on the forums here a while back. It was warning to spec the PSU based on startup power requirements and not normal use. I'm comfortable with that, and don't have a power meter to do in situ testing.

Edit: Spelling

 

[joeinaz]

Three questions will help in recommending a solution:

1. What components of your solution do you already have?
2. What is your budget range or "not to exceed" cost for your FreeNAS solution?
3. Are you considering eBay and or older (not new) hardware?

 

[Lucien]

Three questions will help in recommending a solution:

1. What components of your solution do you already have?
2. What is your budget range or "not to exceed" cost for your FreeNAS solution?
3. Are you considering eBay and or older (not new) hardware?

1. I have the hard drives, and I just placed an order for the X11SSL-F motherboard from Amazon.
2. I'm targeting about $400 for motherboard + CPU + RAM. That's bearing in mind current RAM prices. (Lower is preferable of course.) Everything else is pretty much fixed.
3. Generally no. Shipping here would be a hassle, and there's the lack of warranty. The exception is SSDs. Something like an old Samsung where they're known to be problem free and I could use their utils to check drive health before I use it.

 

[joeinaz]

What is your location? If you just purchased the X11SSL-F on Amazon, it was between $150 and $200 USD. That leaves between $200 and $250 for CPU and RAM. My concern is 8GB of RAM may not be enough for 24TB of disk in a RAID Z2 array. I am having a hard time finding Amazon stock of name brand 8 GB DDR4
ECC UDIMMs. There are brands I never heard of for about $150. The get 16GB, you are looking at $300 for cheap DDR4 memory from Amazon. At this point you could:

1. Go with a low cost Intel G3950 CPU (about $50)
2. Spend a bit extra for 16GB of RAM;
3. Attach your disks to the motherboard SATA ports.

At this point you are looking at about $370 for just the RAM, CPU and fan to complete your project.

 

[diedrichg]

Boot drive: I'm not running mission critical, so I would be willing to consider dual USB flash drives. What would be the worst that can happen if they were to fail? Assuming I kept a copy of the config in a safe place.

Alternatively I may have a ~120 GB SSD in a system I'm not really using any more and could repurpose that as one of my boot drives. Is it possible to mix a 16 GB flash drive with a 120 GB SSD?

Your config backup can live on the NAS drives. Those who are cautious will set up a script to email it to themselves. Assuming the disks survive a pool import after spooling up a new boot drive, once the pool is imported you can just grab the config file directly off your disks.

FreeNAS boot drives can only be used as the boot drive, you can't partition them to use the extra space, it'll use the whole disk. So that shiny new 120GB SSD you just bought will be nothing more than a extremely expensive boot drive with ~ 4-8GB of configuration files and boot environments living on it.

 

[diedrichg]

I have a (now older) Pentium G3420 on an X10 board with 16GB RAM. It is strictly a file server for my immediate family; backing up files and photos with Resilio in a jail and running a sabnzbd jail that sits idle 99% of the time. The limitation I see in the Pentium and lower amount of RAM is file transfer speed. With antivirus suspended, I'll get about 60-75MBps (or is it Mbps? I always get those confused) upload/download between a desktop client and the server. I bet if I had an i3 and 32GB RAM I would get closer to 80-90 during file transfers.

 

[Inxsible]

1. I have the hard drives, and I just placed an order for the X11SSL-F motherboard from Amazon.
2. I'm targeting about $400 for motherboard + CPU + RAM. That's bearing in mind current RAM prices. (Lower is preferable of course.) Everything else is pretty much fixed.
3. Generally no. Shipping here would be a hassle, and there's the lack of warranty. The exception is SSDs. Something like an old Samsung where they're known to be problem free and I could use their utils to check drive health before I use it.

Since you have already bought most of the stuff including the motherboard, it's not a question of trying to show you different options at this point.

The configuration you have chosen should work. It's up to you as to how much money you want to put in it now. You can always keep upgrading the machine slowly.

As an aside, if it were me, I would be A-OK with buying used MB, CPU and RAM but would be wary about buying used storage devices (HDD or SSD) because there could be errors on the HDD and SSDs have a limited number of writes (although that number is large). Putting disks with errors in the pool might be problematic. So I spend my money on used major components but brand spanking new HDD/SSD.

 

[wblock]

FreeNAS boot drives can only be used as the boot drive, you can't partition them to use the extra space, it'll use the whole disk. So that shiny new 120GB SSD you just bought will be nothing more than a extremely expensive boot drive with ~ 4-8GB of configuration files and boot environments living on it.

It will be a solid and reliable boot drive, with lots of space for wear leveling and extra boot environments. Not extremely expensive, even at ridiculously inflated current prices, but that's not a problem if it's already available and not doing anything else.

I would not mirror an SSD with a USB flash drive. The flash drive could be used to store copies of the FreeNAS configuration backup file off-line, although I would also store them elsewhere because USB flash drives are ludicrously cost-reduced.