BUILD Planning new system build and hardware questions

[pc-tecky]

I was hopeful to run FreeNAS as a VM to reduce number of systems, power usage, and costs until I read the thread advising against it. If I did FreeNAS as a VM, I would pass through the drives on an HBA to the VM with SSHD or SSD for VMs and HDDs for ISOs datastore, etc. I currently have FreeNAS (8.?) running with 3x 1.5 TB RAIDZ1 on a Core 2 Duo with 8GB (non-ECC) DDR2 and 3x 3TB HDDs available. Obviously, I need to upgrade. And yes, RAID5/RAIDz1 is overall dead. I'm wanting to plan and design the system to take the best avenue for the least cost, if possible.

What benefit does NAS HDDs and Enterprise HDDs have over other HDDs?

I'm also confused on the plethora of available hardware, namely the CPU performance and platform memory limitations. I have several systems: 2x Core 2 Duo (8GB MAX), 2x Core i5 (32GB MAX), 1x X7DVL-E Dual 1.86 GHz QC Xeon (socket 771, with 8GB ECC FB DDR2, 24GB max; currently my VMware ESXi 5.5 box). Xeon for the longest time has been superior over Pentium's, Core 2 Duo, Core 2 Quad, and Core i3/i5/i7 systems. At this point it seems like Xeon E3 and Core i7 have nearly equal specs and performance, although Core i7 supports greater RAM over Xeon E3, but doesn't support ECC.

Single/Dual Xeon E5-???? up to 1TB of RAM
6-core Xeon X5660/X5670/X5675 Dual CPUs with up to 192GB/288GB/1TB of RAM.
4-core Xeon 5500/5600 series with up to 1TB of RAM.
8-core Xeon D-1540 supports up to 128GB of RAM.
8-core ATOM supports up to 64GB of RAM (32GB more realistic for SO-DIMMs) .
4C/8T Xeon E3-1231 V3 to E3-127? V3 with 32GB of RAM.

$290 - ASRock E3C224D4I-14S Extended mini ITX Server Motherboard LGA 1150 Intel C224 (4x 240-pin) DDR3 1600/1333 & LSI 2308 -- 14x drives (excluding Xeon E3 CPU)

$330 - SUPERMICRO MBD-A1SRi-2758F-O (8-core 64-bit Avoton ATOM) Mini ITX Server Motherboard (4x 204-pin) DDR3 1600/1333

$460 - (yep, that major auction site) 2U - 12-bay chassis with SuperMicro X8DTN+ 2x ~3.0GHz Hex-core Xeon [X5660/X5670/X5675]

or similar 3U or 4U chassis ($200- $500 more)...

 

[anodos]

Enterprise drives have longer warranty and higher MTBF. They also have sas versions. Don't use an i7, i5, anything with an FSB / anything older than nehalem. I imagine there's not a lot of benefit to having hybrid drives. All the ones I've looked at have been unsuitable for NAS use.

 

[Ericloewe]

What benefit does NAS HDDs and Enterprise HDDs have over other HDDs?

NAS HDDs tune down the power-saving features to more properly match NAS workloads and add some features like TLER.
Enterprise drives provide the full feature set, longer warranties and a price to match.

At this point it seems like Xeon E3 and Core i7 have nearly equal specs and performance, although Core i7 supports greater RAM over Xeon E3, but doesn't support ECC.

Not at all. Xeon E3 and Core i7s are literally identical, save for advanced features. Skylake i7s are out already, but the equivalent Xeons aren't out yet - later this year, probably.

CPU-wise, almost any modern Xeon is enough - RAM support is the limiting factor. Ideally, you'll want Sandy Bridge or newer (Nehalem also works, but I'm not familiar with Nehalem SKUs), which means Xeon E3 (32GB max DDR3 for v1-v4, 64GB max DDR4 for v5) or Xeon E5 (RAM support varies, but Xeon E5 v3 16xx support RDIMMs for up to 8x 32GB, so 256GB. Xeon E5 v3 2xxx support LRDIMMs for 64GB per DIMM. Xeon E7 v3 supports insane(ly expensive) amounts of RAM.

 

[Darren Myers]

I have a bunch of 1366 socket servers, theyre a bit power hungry, but preform very well. My FreeNAS server, its plenty powerful for my needs of storage + Plex, i do need to add a bit more RAM, i have 48Gb in there now and I'm probably going to bump that by another 24Gb or so.

As for the harddrives, i have 7 Red's and they preform great, but on the flip side i have 14 2Tb enterprise drives...while the Reds spin at 5400 RPM and have a 3 year warranty, the RE4 (WD Enterprise drives i have) spin at 7200RPM and have a 5 year warranty, which is really nice to have...but at a cost.

On the negative side, the 7200RPM drives do run warmer, so added cooling is needed which is fine...theyre all in a 24 bay chassis anyways so it isnt a real concern if theyre stuck in a tower case it may become an issue without the proper cooling.

Tell us what your long term plans are for the server? Is it for storage of movies,music,misc data,etc? Or is it for storage of VM's and you plan to use iSCSI? little bit of both? Do you plan to do anything with plugins?

 

[Ericloewe]

Reds spin at 5900 RPM

WD Reds are 5400RPM. Seagate NAS are 5900RPM.

 

[Darren Myers]

WD Reds are 5400RPM. Seagate NAS are 5900RPM.

fixed, dont know how i missed that. Thanks

 

[pc-tecky]

Hmm. I may have to re-think this a bit. I have been working with a bargain-basement shoestring budget. As such, I don't have any immediate long-term goals per se. One pressing need is more storage as I am at capacity.

The files already in "storage" are a real mess with duplicates in various places from various sources. A one-time de-dupe might be needed for a "fresh" storage solution - but I don't see de-dupe services as a long-term need (or maybe it will help??). I have a music folder in various states and locations for the great majority of my music collection (some CDs damaged or lost between moves, some MP3 files corrupted over time, and sometimes both are damaged for a given album), documents and photos all over the place from various HDDs of older systems (some still working), hardware drivers, system files - ISOs (Win & Linux), Win SPs, slip-streamed ISOs (win2k-sp2 - winxp-sp3, etc). I wouldn't mind streaming movies (from my movie collection) - for the various devices within my household - win, android, & apple).

FreeNAS is currently configured for SMB/CIFS. I have heard of FTP (insecure), NFS (mostly linux only), and iSCSI. SMB/CIFS and iSCSI might be my best and most promising choices. I have available a free Core2Duo system, a free Core i5 system, 2x 8GB USBs, 1x 120GB SSD, 3x 3TB consumer HDDs, a few consumer 1TB HDDs (2x currently part of ESXi server) to work with. I also have two 20GB/40GB?? infiniband ?? cards (no Infinband/CX4 cables though - idea from a published 'serve the home' project).

No FSB - yep, I'm trying to get away from that. I did a quick search on LGA-1366, some CPUs are Nehalem-based and others are Westmere-based - I don't know if Nehalem is better or equal to Westmere. My current direction was ~$500 for a 2U/12-bay, 3u/16-bay, or 4u/24-bay chassis with dual Xeon 6-core (X5600) Westmere CPUs and 16-/24-/48-GB ECC RAM (as stated in first post).

Enterprise SATA/SAS HDDs, NAS HDDs - good to know about warranty, RPM, TLER, and other aspects. SSHD - not NAS friendly, good to know. SSD or PCIe NVMe device for cache and zlog/metadata
- good to know, most likely out of reach or currently overkill for immediate needs. Boot from USB, but avoid SD media. If upgrading to RAIDz2, get the most drives you can afford, but not necessarily all in the same order or from the same maker (less like to have massive snow-ball failures - or it that just a myth?).

If working with a shoestring budget, what would you buy first, if anything? HDDs, HBA(s) & expanders, CPU, RAM, MoBo? How would you prioritize? MoBo with LSI 2308 8-port SAS? Embedded Atom CPU? Embedded Xeon D-1540 CPU? Xeon E3-1200 v3/v4 (LGA-1150) or v5 (LGA-1151) CPU?

 

[pc-tecky]

Well, I got some newer hardware. I didn't see anything like MEMTEST etc. But now I see that is even limited. I heard that the (on-board) LSI 1068e controller will not support 3TB+ HDDs.

I'm confused with documentation - just how many USB drives do I need? One USB for install - ok. One -or- two USB for system??

 

[Robert Trevellyan]

I heard that the (on-board) LSI 1068e controller will not support 3TB+ HDDs.

You can connect larger drives, but the 1068E won't see anything beyond about the first 2.2TB.

how many USB drives do I need?

At minimum, and to be pedantic, one for the OS, if you install from a CD.

Assuming you install from USB, two is the minimum.

Many people take advantage of the fact that the OS now resides on a ZFS filesystem to mirror their boot device, which means three.

 

[pc-tecky]

Thanks for the reply. Err, well, that's new. I installed 9.3 from Lexar 8GB stick onto PNY 16GB (micro, sized like Sandisk USB Cruzer Fit), and then upgraded to 9.10 stable online, then migrated 3x 1.5TB consumer drives from prior Core2Duo with 8GB (maxed out) running 8.2?. I'm guessing this is an issue with the boot media or bad memory. How do I trouble shoot and resolve? Should I start over with yet another fresh install? How to test? I was fixing to add 3x 3T consumer drives, if full capacity can be detected by system.

• CRITICAL: May 2, 2016, 8 p.m. - The boot volume state is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.
• WARNING: May 1, 2016, 1:47 p.m. - ZFS filesystem version is out of date. Consider upgrading using "zfs upgrade" command line.

 

[Robert Trevellyan]

Maybe the PNY stick is bad or flaky. Try a different stick. If necessary, just swap them, 8GB is ample capacity for a FreeNAS boot device (mine is 4GB).

 

[pc-tecky]

I need to see if I have another USB drive to test with. Is there a way to save and restore user and configuration data? In the mean time, I just got done with yet another fresh install and upgrade on PNY 16GB USB micro-sticks (brand new). Might this be related to: https://bugs.pcbsd.org/issues/14317.

Just added second PNY 16GB USB micro-stick and currently re-silvering. Hopefully, this:

• CRITICAL: May 3, 2016, 11:04 p.m. - The boot volume state is ONLINE: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state.
• WARNING: May 3, 2016, 10:34 p.m. - ZFS filesystem version is out of date. Consider upgrading using "zfs upgrade" command line.

doesn't go to this (from above):

• (red) CRITICAL: May x, 2016, x:xx a/p.m. - The boot volume state is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.
• (yellow) WARNING: May x, 2016, x:xx a/p.m. - ZFS filesystem version is out of date. Consider upgrading using "zfs upgrade" command line.

Think I'll run memtest as well.

 

[Bidule0hm]

Just save your config file (GUI --> System --> General --> Save Config button) and you'll not worry anymore ;)

 

[rogerh]

And you do know that you mustn't upgrade the boot filesystem? This "out of date" warning isn't shown by more recent FreeNAS versions.