Will it FreeNAS - My First Build

linus12

Explorer
Joined
Oct 12, 2018
Messages
65
I've pretty much secured funding for my project and am in the configuration stage. But before I pull the trigger I thought I'd run it by you all to see a) if what I am building will work, and b) if there are any suggestions.

My goal (besides building something I can show off and impress my friends who don't have a server ;) ) is to build a server that will last at least 5 years, hold all the stuff that is ready to go on it, and be expandable enough to add more drives and other software as my needs expand.

Initial use will be to offload my ripped Blu-Ray and DVD movies currently housed on multiple systems: Current size 4TB, eventual expected size on server 10TB
Along with the family's Music collection. Current size 45GB, eventual expected size on server 1TB

Along the way I'm sure the photo collection (Currently 2TB and growing) will make it's way there
As well as being used for some backup purposes.
Each family member (4 of us) is also looking for a couple of TB of storage for freeing up space on laptops.

My Thought process was to start out robust on the MB & CPU, Initial steps for RAM and Drive Space, and then expand RAM and Drive Space as I get more comfortable administering the server and the others get more comfortable using it.

To that end, here is my proposal:
  • Motherboard:
    • SuperMicro X11SSH-CTF-O
    • OR
    • SuperMicro X11SSH-F
  • CPU: Intel Xeon E3-1275-V6 (includes HD Graphics P630 so a video card is not needed)
  • RAM: 32 GB (Getting tough to find 16GB sticks on SuperMirco's CVL)
  • Hard Drives:
    • 2 x Seagate IronWolf 10TB NAS Hard Drive 7200 RPM 256MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive ST10000VN0004
    • OR
    • 2 x WD Red 10TB NAS Hard Disk Drive - 5400 RPM Class SATA 6Gb/s 256MB Cache 3.5 Inch - WD100EFAX
  • Boot Drive: Either an M.2 drive from the CVL or a SSD from the CVL. 512GB Capacity
  • Network card: Built into Motherboard
  • Hard Disk Controller: As I understand it, the X11SSH-CTF already has a LSI3008 built in so I'm leaning towards that
Hard drives will be implemented using Raid 1. I'm looking for fault tolerance and read speed. Full backups will be kept offsite. But most of the data will be write once, read many. But should one of the drives go, I want to be able to limp along for a few days to replace it if I am out of town.

After initial setup, an additional 2 hard drives (10TB each in Raid 1) will be added, and finally an additional 2. Total expected to be 6 drives providing 30TB of space.

After initial setup, and additional 2 sticks of 16GB of memory will be added for a total of 64GB (Max that the E3-1200 series can handle)

Of course the PSU will be sized accordingly OR will be upgraded as drives are added.

Cooling will be provided by an All-in-one loop for the CPU and fans for the drives.

The case is still undecided, but I may have access to CoolerMaster HAF X case. While oversized, it will easily handle the number of drives and the fans to keep them cool with minimum noise. My son keeps saying I should get a 2U or 3U rack mount case and put it under the laser printer... but I'm worried about keeping the drives cool. (The system will be in my home office which is not know for being super chilly, so I know airflow will be a big consideration.

I hope I have provided a somewhat clear build path and I am open to most suggestions. While this is not a "money is no object" build, I do want a system that will last, so I'm willing to spend a few more bucks now so I don't have to scrap and rebuild from scratch in a few years.

Looking forward to all of your recommendations.
 

linus12

Explorer
Joined
Oct 12, 2018
Messages
65
Having just re-read this, I think I used the wrong terminology for the drives. It's not really Raid 1, but rather "mirrored pairs". Again I'm looking for that redundancy in the event of a single drive failure. If both fail, it's going to be a long rebuild as I a pull the files off the back up!
 

IQless

Contributor
Joined
Feb 13, 2017
Messages
142
Boot Drive: Either an M.2 drive from the CVL or a SSD from the CVL. 512GB Capacity
The boot drive can not be used for anything else than the os, so this would be overkill by far :P Maby go for a cheap SATA SSD or two, and use the money elsewhere in the build? Just my two cents :)
 
Joined
Jul 2, 2019
Messages
648
I picked up two Kingston 240GB SSDs on Amanzon for CDN$25 each...
 

JohnnyGrey

Dabbler
Joined
Jul 1, 2017
Messages
45
SuperMicro motherboards have a really low powered GPU built in, so getting a cpu with onboard graphics isn’t necessary. Besides, FreeNAS is meant to be headless, so other than initial setup, you’ll rarely need to have any peripherals hooked up.
 
Joined
Jul 2, 2019
Messages
648
Another thought: Have you considered purchasing a used server? Plenty of good ones on eBay. Many of us have gone the route of used servers.
 

linus12

Explorer
Joined
Oct 12, 2018
Messages
65
The boot drive can not be used for anything else than the os, so this would be overkill by far :p Maby go for a cheap SATA SSD or two, and use the money elsewhere in the build? Just my two cents :)

I picked up two Kingston 240GB SSDs on Amanzon for CDN$25 each...

Thanks for the info on the Boot Drive. Yes I did think about a cheap SSD, and I will reconsider it. I was thinking more along the lines of less cabling and thus less blockage for air flow along with the minimal increase is heat. (My current SSDs are warm, not hot, to the touch, but still cooler than the hard drives on my PC.

SuperMicro motherboards have a really low powered GPU built in, so getting a cpu with onboard graphics isn’t necessary. Besides, FreeNAS is meant to be headless, so other than initial setup, you’ll rarely need to have any peripherals hooked up.

I did see the VGA port didn't know how if it really worked OK. Yes, my plan is to run it headless, but you know... sometimes when you've got network problems, connecting a Monitor, keyboard, and mouse makes life so much easier. -- I run my Raspberry PI headless, but have an old converted laptop display in a picture frame that I keep connected just in case; that along with the extra keyboards and mice I have on hand have saved my bacon more than once when the router or switch started acting up. (Of course I've since replaced them.....) OK, guess that means I can up the CPU to the Xeon E3-1280 V6 and get an extra 0.1 GHz on the Base frequency ;)
 

linus12

Explorer
Joined
Oct 12, 2018
Messages
65
Another thought: Have you considered purchasing a used server? Plenty of good ones on eBay. Many of us have gone the route of used servers.
Of course I have! But you know, I haven't designed, sourced, or built a computer in almost 18 months and I really like knowing where everything is and how it's all put together. That and the bragging rights of being able to say "I built that!".
 

linus12

Explorer
Joined
Oct 12, 2018
Messages
65
Any other comments on the rest of the build? Like choosing the MB with the built in LSI3008 vs buying an additional controller?
I realize that I'll be limited to 6 drives on that controller and that if I want to add more, I'll have to pick up an LSI card.​

What about the memory?
  • Is 32 going to be enough to start? or Should I max it out at 64 now?
  • Does speed really make a huge difference here?
    • I know there is more to it than just the speed, but the vendor list only list 3 three: 2133, 2400, and 2666. Availability seems to be biggest Issue.
  • The Motherboard seems to be about 3 years old. Is this going to be an issue down the line? I understand that whenever you purchase something, something better is immediately released. But with the VCL memory being is short supply, I'm wondering if I should hold off 3 months or so if the SuperMicro X12 models are on the horizon....
 

anmnz

Patron
Joined
Feb 17, 2018
Messages
286
I'll be limited to 6 drives on that controller and that if I want to add more, I'll have to pick up an LSI card.
No. A SAS controller can run many many many more drives than that. You just need to add appropriate SAS expanders. (Or plug into a suitable backplane of course.)

Is 32 going to be enough to start? or Should I max it out at 64 now?
I don't think you mentioned any memory intensive use case. 32GB sounds plenty.

Does speed really make a huge difference here?
RAM speed is unlikely to make any observable difference to the performance of your NAS. It just won't be the bottleneck.

The Motherboard seems to be about 3 years old. Is this going to be an issue down the line? I understand that whenever you purchase something, something better is immediately released
Not really. For my money the last really dramatic improvements In the sort of x86 server hardware we are talking about were the various power efficiency improvements in Sandy/Ivy Bridge over the previous Intel generation. Changes since then have been steady but very incremental and three years is nothing you'll even notice in practice.

Which is why I'd go with @Newfoundland.Republic and say to seriously think about used hardware. Doesn't need to be a whole used server. Just used parts can save you a heap of money with no performance / compatibility downside you'll ever really notice.
 
Joined
Jul 2, 2019
Messages
648
One other thing to consider is that make sure that you have IPMI built in. Most Supermicro boards seem to have IPMI. Great for remote setup, etc.! No need to go into the room with the server :smile:
 

JohnnyGrey

Dabbler
Joined
Jul 1, 2017
Messages
45
Sorry if I quoted these out of order!

I did see the VGA port didn't know how if it really worked OK...

It worked fantastic for me. Just make sure you have a monitor with a VGA input. It's a really low powered GPU, and only has like 32mb of VRAM or something, but most of what you'll be looking at will be 640x480 or some similarly low resolution anyways.

One other thing to consider is that make sure that you have IPMI built in. Most Supermicro boards seem to have IPMI. Great for remote setup, etc.! No need to go into the room with the server :)

I believe all of the X11 motherboards have IPMI. With that said, I would recommend updating the IPMI firmware, assuming it ships with an older firmware. Older versions used Java for the remote KVM, whereas newer versions switched to the much better HTML5. I think that remote virtual media might be broken in HTML5 though. Don't quote me on that part.

I realize that I'll be limited to 6 drives on that controller and that if I want to add more, I'll have to pick up an LSI card.

What about the memory?
  • Is 32 going to be enough to start? or Should I max it out at 64 now?
  • Does speed really make a huge difference here?
    • I know there is more to it than just the speed, but the vendor list only list 3 three: 2133, 2400, and 2666. Availability seems to be biggest Issue.
  • The Motherboard seems to be about 3 years old. Is this going to be an issue down the line? I understand that whenever you purchase something, something better is immediately released. But with the VCL memory being is short supply, I'm wondering if I should hold off 3 months or so if the SuperMicro X12 models are on the horizon....

  • Remember, the controller provides additional connections over the eight provided by the c236 chipset. The LSI 9300-8i card uses SFF8643 connectors. it looks like the connectors on this motherboard are indeed the same. Someone else should confirm. Assuming you pick up some breakout cables, out of the box you have connections for 16 drives.
  • Maxing out your memory is up to the relationship you have with your wallet. I have an X11 board myself, and while I have more drives, they are much smaller drives, and 16gb has still served me well. 32gb is probably plenty for now.
NOTE: Another thing to keep in mind is you'll have to flash the controller to IT mode. It appears to ship with an IR mode firmware. From a brief Google search, flashing it most likely voids any warranty with SuperMicro. I could be wrong on that.
 
Last edited:

linus12

Explorer
Joined
Oct 12, 2018
Messages
65
Well it took longer than I expected, but I've sent the Purchase order in for the first set of the new build. Thanks to help here, this is what I'm looking at:

To that end, here is my proposal:
  • Motherboard: SuperMicro X11SSH-CTF-O
  • CPU: Intel Xeon E3-1280-V6 3.9 Ghz
  • RAM: Kingston D749E DDR4-2400 16GB ECC Server Memory
Will look further when the above arrives:
  • Boot Drive: Small SSD, probably pick up what's on sale
  • Hard Drives: (may have a line on getting the WD drives cheap, so may end up going that way!)
    • 2 x Seagate IronWolf 10TB NAS Hard Drive 7200 RPM 256MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive ST10000VN0004
    • OR
    • 2 x WD Red 10TB NAS Hard Disk Drive - 5400 RPM Class SATA 6Gb/s 256MB Cache 3.5 Inch - WD100EFAX
I'm sure I'll be over in the Installing Help area soon!!!
 

MikeyG

Patron
Joined
Dec 8, 2017
Messages
442
On the X11SSH-CTF watch out for the M.2 slot. I believe you can only fit a 60mm drive in there, not a normal consumer 80mm M.2 nvme drive. The nvme drive will work, but the screw layout isn't adjustable. Found that out the hard way on mine, and it sadly goes unused to this day. You'd have to get some electrical tape or something to make a normal drive work in it.

Otherwise, great board. The 3008 on mine was set to IR mode and did not flash for a while - passed drives through correctly as long as I didn't create a hardware RAID with the onboard utilities.
 

linus12

Explorer
Joined
Oct 12, 2018
Messages
65
On the X11SSH-CTF watch out for the M.2 slot. I believe you can only fit a 60mm drive in there, not a normal consumer 80mm M.2 nvme drive. The nvme drive will work, but the screw layout isn't adjustable. Found that out the hard way on mine, and it sadly goes unused to this day. You'd have to get some electrical tape or something to make a normal drive work in it.

Otherwise, great board. The 3008 on mine was set to IR mode and did not flash for a while - passed drives through correctly as long as I didn't create a hardware RAID with the onboard utilities.
Thanks for the heads up. I originally was planning to use the M2 slot for the boot drive but was talked out of it (See thread above). So yea, it will go unused for now. Really chose this one for the embedded 3008. Already have the instructions from SuperMicro to flash it. Seems there was a lot of misinformation about that and they were really quick to assure me that it can be done and was fully supported.
 
Top