first Nas build - ready for takeoff

[JPatrick]

Hello all,

This will be my first freenas build; I wanted to make sure everything checked out before I committed and completed the build (already have the WD Red drives and Case). I'd like the box to be able to serve as a centralized home file server/media library, and run plex with a max of 3-4 concurrent transcoding streams.

Motherboard: SUPERMICRO MBD-X11SSM-F-O Micro ATX
CPU: Intel Xeon E3-1230 v5 SkyLake 3.4 GHz [BX80662E31230V5]
Ram: (2x) Samsung 16GB ECC DDR4 UDIMMs [M391A2K43BB1-CPB]
HDD: (8x) WD Red 3TB 5400 RPM HDDs [WD30EFRX], to be run in RaidZ2
Boot Device: (2x) SanDisk Cruzer Fit 16GB USB 2.0 Low-Profile Flash Drives [SDCZ33-016G-B35]
PSU: EVGA SuperNOVA 650W 80+ gold [220-G2-0650-Y1]
Case: Fractal Design Define R5

The questions:

1) (The obvious one): Any glaring problems?

2) I suspect this may be somewhat overbuilt for my needs, anywhere you think I can save a bundle while still accomplishing the described end-use? I think I could probably shave ~100-150$ off the build without straying too far from the recommended components list, but I feel the savings come at considerable costs in future upgradability, power efficiency, and general value.

3) As a possible alternative to this build I've toyed with the idea of going with an ASRock C2750D4I mobo as a lower power & noise option, but the relatively low PassMark score of ~3800 seems to indicate I might run into problems with multiple concurrent transcodes. Could anyone with experience running plex on this board chime in on their experiences? (FYI: the media files to be transcoded are uncompressed mkv bluray rips).

 

[Ericloewe]

anywhere you think I can save a bundle while still accomplishing the described end-use?

Start with 16GB of RAM, expand later if/when needed.

3) As a possible alternative to this build I've toyed with the idea of going with an ASRock C2750D4I mobo as a lower power & noise option, but the relatively low PassMark score of ~3800 seems to indicate I might run into problems with multiple concurrent transcodes. Could anyone with experience running plex on this board chime in on their experiences? (FYI: the media files to be transcoded are uncompressed mkv bluray rips).

Performance should be fine - reliability is far from it, though.

 

[Jailer]

For your intended usage your component selection will serve you well. Only thing I'm not all that familiar with is the power supply. @Ericloewe is the resident PSU guru so maybe he can chime in on the quality of that unit.

 

[Ericloewe]

For your intended usage your component selection will serve you well. Only thing I'm not all that familiar with is the power supply. @Ericloewe is the resident PSU guru so maybe he can chime in on the quality of that unit.

It's fine. The stamped steel grille that EVGA loves so much can lead to needless noise, though - Seasonic and Corsair do it right and use a wire grille.

 

[joeschmuck]

Agreed, you could save about $90 (what I found it for earlier today) by using only one 16GB stick of that RAM. If you find that you need more RAM then you can add it later.

So you are looking at approximately 16TB of storage, minus 3.2TB (20%) to retain good pool speed equals approx. 12.8TB of actual storage. If this is more than you need then you could save some more money by reducing your drive count or reducing the capacity of the drives. If this isn't enough storage then you could increase the drive capacity and/or change the drive count.

Pay attention to the deals this weekend!

 

[JPatrick]

Thanks guys!

Based on your input, I've changed the PSU to a Corsair RM750X, which is both cheaper than the EVGA unit I had originally planned to go with (75$ after rebate, vs 90$) and (oddly enough) cheaper right now on newegg than the 650w version.

Consensus seems to be to toss one of the 16GB sticks: if I can get away with this, awesome (90$+ in the bank), but everything I have read has been pretty pointed about not skimping on RAM with ZFS. Following the 8GB + 1GB/TB Disk Space rule, I figure I need a minimum of 32GB (8GB + 24GB). Can I really make do with half that? What kind of risk of data loss am I looking at? The media library is non critical (and I plan on backing up everything else), but re-ripping all those movies would probably drive me bonkers.

joeschmuck: the size of the pool is actually about 8x larger than I currently need, but I hope to grow into it! :D I figure in 3-4 years I could very realistically expect to have filled 75% of the total usable space, at which point I can swap out the drives for larger ones. An 8 drive pool seems like a good way to minimize loss due to parity, and the 3TB WD Reds seem to offer the cheapest $$$/usable TB in the line.

Also, did I do my maths wrong? I figured with 8x3TB drives in raidz2 I would have 6 usable drives (2 for parity) for a total of 18TB. Assuming the 20% (3.6TB) overhead to retain speed, that would leave me with ~14.4TB. I'm probably missing something stupid obvious...

 

[Ericloewe]

Also, did I do my maths wrong? I figured with 8x3TB drives in raidz2 I would have 6 usable drives (2 for parity) for a total of 18TB.

Yes, those drives (and all others) are 3* 10^12 bytes. That's roughly 0.909 of the proper 3*2^40 bytes

What kind of risk of data loss am I looking at?

Same as with 32GB. It's a rule of thumb, nothing more.

 

[brando56894]

Don't even think about going anywhere near the C2750D4I unless you want a dead system in about 12-24 months.

 

[joeschmuck]

Consensus seems to be to toss one of the 16GB sticks: if I can get away with this, awesome (90$+ in the bank), but everything I have read has been pretty pointed about not skimping on RAM with ZFS. Following the 8GB + 1GB/TB Disk Space rule, I figure I need a minimum of 32GB (8GB + 24GB). Can I really make do with half that? What kind of risk of data loss am I looking at? The media library is non critical (and I plan on backing up everything else), but re-ripping all those movies would probably drive me bonkers.

It would be good to get rid of all the references to that rule of thumb because doesn't fit well here in our little FreeNAS world. 8GB RAM is the minimum you need to run FreeNAS and 16GB gives you more than enough room for the tasks at hand. The RAM has nothing to do with the ability to keep your data safe, other than it must be ECC RAM.

the size of the pool is actually about 8x larger than I currently need

That is very good. Rarely do I see someone plan ahead without pushing them in that direction.

As a note about ripping movies, this is just my own advice but I rip the movies that I will see multiple times or ones the grand kids will watch over and over (dirty little fingers destroy DVDs). Just bought Finding Dory at Walmart for $15 (DVD/BluRay/Digital Download set) and I'll be ripping it for the grand kids, that digital download would be nice if I could actually download a local digital copy, save me from ripping it. But, just building up a digital library to show it off is a waste of space and eventually you will find out that you have ripped so many movies that you will never watch again and start deleting those when you run out of space. A typical rip for me is about 1.5-1.7GB, a BluRay rip is almost 4X that and honestly, while I can tell the difference between the two, it's not that noticeable unless you actually are examining the video content for flaws. I guess if you have a 90" TV that could make a difference too but I have a 55" TV.

For my FreeNAS machine, I have a massive movie library of 265GB, that's a few movies (maybe over 100 movies). The rest of my storage is photos (~60GB), Music (~57GB), Software and other data (~175GB), and the big dent is Computer Backups (~3.9TB). Everyone in the house has a computer and since I really hate rebuilding from scratch and listening to bellyaches about lost data, I routinely backup these computers. The backups go back 3 months. Its a lot of data but this is the data which is by far the most important to me. It's the entire reason I built a NAS in the first place. The fact that it allows me to copy my DVD movies (and a few BluRay) and run Plex is the added bonus. For the money I just don't see FreeNAS as a reasonable media server, it's too expensive. But I'm old, a member of the Baby Boomers (barely) and you are probably a Millennial so there is a gap. My thinking is a dying breed. Please take my comments as ramblings from an old man which might have a little bit of wisdom.

 

[brando56894]

A typical rip for me is about 1.5-1.7GB, a BluRay rip is almost 4X that and honestly, while I can tell the difference between the two, it's not that noticeable unless you actually are examining the video content for flaws. I guess if you have a 90" TV that could make a difference too but I have a 55" TV.

Agreed, even though I have a 40" 1080P TV I store the majority of my stuff in 720P because I can't tell the difference between the two 90% of the time. There are very few files that I have in 1080P, mostly visually stunning movies and nature documentaries (like the current Planet Earth II). I plan on getting a 4K TV soon so the things I have in 1080P will be switched to 4K. I actually did some research a while ago on the difference between 720i, 720P and 1080P (4K wasn;t out yet) and it all depends on how close you are to the screen and how big the screen is. At 2 feet away from a 40" screen you can tell the difference between 720i, 720P, 1080P. If the screen size stays the same, the further you move away, the less the resolution matters. At something like 20 feet away away from 40" screen you can't tell the difference between 480P and 1080P.

 

[JPatrick]

Yes, those drives (and all others) are 3* 10^12 bytes. That's roughly 0.909 of the proper 3*2^40 bytes

Doh

Same as with 32GB. It's a rule of thumb, nothing more.

It would be good to get rid of all the references to that rule of thumb because doesn't fit well here in our little FreeNAS world. 8GB RAM is the minimum you need to run FreeNAS and 16GB gives you more than enough room for the tasks at hand. The RAM has nothing to do with the ability to keep your data safe, other than it must be ECC RAM.

16GB it is!

Don't even think about going anywhere near the C2750D4I unless you want a dead system in about 12-24 months.

Thanks for the heads up, I'll go with a xeon board

As a note about ripping movies, this is just my own advice but I rip the movies that I will see multiple times or ones the grand kids will watch over and over (dirty little fingers destroy DVDs). Just bought Finding Dory at Walmart for $15 (DVD/BluRay/Digital Download set) and I'll be ripping it for the grand kids, that digital download would be nice if I could actually download a local digital copy, save me from ripping it. But, just building up a digital library to show it off is a waste of space and eventually you will find out that you have ripped so many movies that you will never watch again and start deleting those when you run out of space. A typical rip for me is about 1.5-1.7GB, a BluRay rip is almost 4X that and honestly, while I can tell the difference between the two, it's not that noticeable unless you actually are examining the video content for flaws. I guess if you have a 90" TV that could make a difference too but I have a 55" TV.

You're definitely right about this filling a "want" rather than "need" niche. That said, the box will be a do-it-all machine for a household, including backups for 4 PCs and media off a bunch of mobile devices. I could probably save on the storage...but I'm a digital packrat, and I imagine I would be wanting more sooner rather than later.


Regarding the build components....The board I was originally looking to buy is now out of stock and I'm considering a few alternatives:

• Supermicro MBD-X11SM-O - Basically the same board as the MBD-X11SSM-F-O, minus the IPMI capability. Could anyone chime in on whether they actually find the functionality useful? Or whether I should save a few dollars by going with a board without it?
• Supermicro MBD-X11SSH-F-O - Same as the X11SSM-F-O (including IPMI), minus a PCI slot
  
• Supermicro MBD-X11SSH-LN4F-O - +Quad ethernet

thoughts?

 

[joeschmuck]

While IPMI is not a must, it sure can be convenient. If you are looking to go for a headless system (no monitor or keyboard) then IPMI helps a lot. You can mount an ISO image via IPMI, same as connecting up a virtual CD drive. If you are a tinkerer, get IPMI. If you are a set it up and forget it kind of person, you won't need it.

One less PCI-E slot will be fine for you. The main reason for the PCI-E slots in a FreeNAS system is for adding an HBA card so we can add more hard drives or adding in new or additional connectivity like maybe a 10G Ethernet or fiber. Something which gets neglected which discussing PCI-E slots is the bandwidth of each slot. Just because a slot is x16 long doesn't mean it's a true x16 slot, same with the x8 slots. On the X11SSM-F the x16 slot is only a x8 and only one of the three x8 slots are actually x8, the other two are x4. Read the user manual carefully.

Quad Ethernet ports are not needed and you are apt to only use one port anyway. Quad ports would be fine if you wanted to go build a full fledged server, like ESXi or similar.

 

[brando56894]

I could probably save on the storage...but I'm a digital packrat, and I imagine I would be wanting more sooner rather than later.

You're not alone, it's a sickness that a lot of us suffer from, and this is our support group :D Regarding IPMI it is really neat if you've never used it before. When I got my C2750D4I I didn't have a case yet and resorted to manually shorting the pins to turn it on and used a flash drive connected to USB to install freenas because I had no idea what IPMI was hahaha Now even though my server sits directly under my desktop (it's mini ITX) I still resort to IPMI because it's easier than swapping cables. The only thing that's sucks is the web front-end for it is a Java applet, so out of IE, Chrome and Firefox, Firefox is the only one that supports it. There is a Windows front end for it, along with command line and probably GUI front ends for Linux and OS X also.

 

[Chris Moore]

Regarding the build components....The board I was originally looking to buy is now out of stock and I'm considering a few alternatives:

• Supermicro MBD-X11SM-O - Basically the same board as the MBD-X11SSM-F-O, minus the IPMI capability. Could anyone chime in on whether they actually find the functionality useful? Or whether I should save a few dollars by going with a board without it?
• Supermicro MBD-X11SSH-F-O - Same as the X11SSM-F-O (including IPMI), minus a PCI slot
  
• Supermicro MBD-X11SSH-LN4F-O - +Quad ethernet

thoughts?

I apologize in advance, I am a bit long winded.

As for your build, it is a few generations newer than mine with the performance increases that implies, but there are some similarities in the hardware that make me confident you will get the results you are looking for. I would say that sometime in the future, you might want to have an actual server chassis with front drive trays to make changing drives easy.

I guess that I am late to respond on this. I would say that IPMI is almost a must. The first several NAS builds that I did were using old workstations (Dell Precision) that were built with Xeon processors and ECC memory from the factory. They worked great, but they didn't have enough drive bays, and no IPMI. Once I got clued into IPMI, I will never go back. I have four SuperMicro servers running right now and an HP. The HP and the oldest SuperMicro don't have IPMI and (after getting used to not doing it) it is such a hassle to walk into the back room and plug a monitor and keyboard in so I can see a text console.
There are some good videos on YouTube telling about remote management with IPMI and you might want to look at some to help you understand.
With IPMI, you can plug in the electric and network connections and then go sit at your desk in the other room and remotely mount the ISO file via the network, then power the system on and view or edit the BIOS (UEFI) settings to set your boot order, then boot to the remote ISO as if it were a local CD in the system. From there you can step through the whole setup process, all without ever connecting a keyboard or mouse or monitor or CD drive to the NAS and you did it all from another room over the network.
You can do without IPMI, but once you use it, you won't want to do without it...

PS. More expansion slots is usually better, in my opinion.

 

[28061]

+1 for IPMI. It's a life saver for me. Aside from the convenience, my rack server doesn't have an optical drive, so the ability to remotely mount ISO files is brilliant.


Sent from my iPhone using Tapatalk

 

[Stux]

So, the coolest toy since sliced bread.

Also means you may be able to write custom fan controller scripts ;)

(See sig)

 

[JPatrick]

You can mount an ISO image via IPMI, same as connecting up a virtual CD drive. If you are a tinkerer, get IPMI. If you are a set it up and forget it kind of person, you won't need it.

The ability to mount an ISO was one I wasn't even aware of, awesome (learnsomethingnew)

Regarding IPMI it is really neat if you've never used it before.

I would say that IPMI is almost a must.

+1 for IPMI. It's a life saver for me.

So, the coolest toy since sliced bread.

IPMI it is! Going with the XLL-SSH-F

There are some good videos on YouTube telling about remote management with IPMI and you might want to look at some to help you understand.

Definitely will do

I read in an older thread that the skylake processors lack EHCI support, and since Freenas lacks XHCI drivers, that means no booting off USB on a skylake powered board. Was that fixed in the 9.10 release?

 

[Ericloewe]

Was that fixed in the 9.10 release?

Yes.

 

[Chris Moore]

I read in an older thread that the skylake processors lack EHCI support, and since Freenas lacks XHCI drivers, that means no booting off USB on a skylake powered board. Was that fixed in the 9.10 release?

I know that some people will disagree with me on this, but I use a mirrored pair of 40GB laptop drives for my boot pool. That way I can run diagnostics against them and monitor their health, which you can't do with USB. They don't use much power or generate much heat and the reliability has been better than with USB. I have had a USB stick go bad on me, which was a pain to recover from because I didn't have a current backup on my running config AND I was only using a single USB stick instead of a mirrored pair of them. I made some mistakes, but even without those mistakes, you still have no way to monitor the health of your USB stick.
The laptop drives I have been using are 2 years in and have had no faults at all.
The speed of the boot drives is not really an issue either.
Something to think about: http://www.ebay.com/itm/291957031444

 

[joeschmuck]

I know that some people will disagree with me on this, but I use a mirrored pair of 40GB laptop drives for my boot pool.

Don't get me started! Laptop hard drives on FreeNAS. Those had better been Free laptop hard drives at least. ;)
Seriously, I see nothing wrong in using laptop hard drives, they are better than USB flash drives. I have a few laptop hard drives and I use them for various things.