X11SSM-F FreeNAS Build Verification

[kernalzero]

So I've run this through PCPartPicker and a friend who has a FreeNAS box already running but I wanted some final verification before I take the plunge. Budget is $2k. Use cases: Plex server (4-5 streams simultaneously max, regular load 2), TimeMachine back up for 2 Macs, general file server, NextCloud/OwnCloud, 3-4 VMs (windows 10 and a few linux flavors). I'm expecting this build to last roughly 10 years and expect a couple drive failures along the way.

Linked Database with URLs

Build:

• SuperMicro X11SSM-F
• Intel Xeon E3-1225 V5
• Crucial 16GB MTA18ASF2G72AZ-2G1A1 ECC x2 (32GB total)
  
• WD Red 4TB x6 (24TB total, RAID-Z2 14.04TB)
• Mirrored PNY 120GB SSDs x2 (I couldn't find cheaper, smaller drives to run OS)
• Fractal Node 804
• SeaSonic 650 (Gold)
• Eaton 5S700LCD UPS

Right now this is slightly under budget at $1918.58 (as of 5/13/2018).
A few acknowledgements: mirrored SSDs for the OS is overkill, so is 120GB but the price between the SSD and a 16GB DOM is basically nothing. 650w PSU is probably slightly oversized but gives me room to grow.

Questions:

• Should I drop down to 16GB of DDR4 and invest the extra $$ on a stronger Xeon or is 32GB appropriate for my use cases?
• I think I'd prefer a V6 Xeon but don't want to risk getting a mobo without an updated firmware - this sounds like a pain in the ass as I would need a V5 to update firmware. Is this a real concern - it feels like one?
• Any other suggestions for the build?

Thanks for your feedback.

 

[Dice]

Crucial 16GB MTA18ASF2G72AZ-2G1A1 ECC x2 (32GB total)

The parts picker is apparently useless.
You'll need a unbuffered ECC memory for this board.

A few acknowledgements:

fine.

Q1) the 1225 is gimped without HT. Look for at least the 1230v6. There are diminishing returns when looking into higher end models on the same platform. RAM can be upgraded easily. Stick with 16GB sticks.
Q2) I recon it <was> a real concern. It should not be now.
Q3) No reason to run mirrored SSD for the OS. One 40Gb-isch would be plenty. Add another one to host your jails/vm's etc.
Q4) To hope for 10 years life is in my opinion not reasonable without looking at the very top end of current products. That is the ludicrously expensive top of the line stuff from intel.
However, if you'd find yourself getting this to last 10 years in <any sort of use case related to fileserving> that's completely fine. Expecting it to do 'what can presently be done while cutting edge' years from now is to expect too much. Either in terms of raw power, or in terms of reasonability with regards to expected power efficiency savings.

 

[kernalzero]

The parts picker is apparently useless.
You'll need a unbuffered ECC memory for this board.

Micron makes parts for Crucial right? This is the part number off supermicro's memory compatibility site. on the Amazon link it also says "Crucial Memory CT16G4WFD8213 16GB DDR4-2133 Unbuffered ECC UDIMM Retail".

Thanks for the other advice. I'll check out the V6s. I'll also dig into the memory piece more. 10 years may be a bit much to expect for a $2k server but I'd like to see it last at least half that excluding what could happen over the next decade. I can't expect it to handle streaming 5 4k fully immersive virtual reality environments if that's where things are heading. Over time I expect I'll be offloading some things for sure.

 

[kernalzero]

Crucial Link to the memory in question

Crucial 16GB DDR4-2133 ECC UDIMM CT16G4WFD8213

• Form Factor: EUDIMM
• Total Capacity: 16GB
  
• Specs: DDR4 PC4-17000 • CL=15 • Dual Ranked • x8 based • Unbuffered • ECC • DDR4-2133 • 1.2V • 2048Meg x 72

Good call on the E3-1230. Both V5/V6 both have HT - V6 is way cheaper too.

 

[Dice]

• Specs: DDR4 PC4-17000 • CL=15 • Single Ranked • x4 based • Registered • ECC • DDR4-2133 • 1.2V • << link in OP :)

 

[kernalzero]

• Specs: DDR4 PC4-17000 • CL=15 • Single Ranked • x4 based • Registered • ECC • DDR4-2133 • 1.2V • << link in OP :)

Yeah - Amazon is hit or miss... The part number seems like the truth but if you can't get the description right can they get the right part # in the box... i'll update my parts list and get my memory from crucial.com. $222.99/stick there.

 

[Inxsible]

I think I'd prefer a V6 Xeon but don't want to risk getting a mobo without an updated firmware - this sounds like a pain in the ass as I would need a V5 to update firmware. Is this a real concern - it feels like one?

Use IPMI to upgrade the BIOS. You don't need a CPU then.

 

[kernalzero]

Use IPMI to upgrade the BIOS. You don't need a CPU then.

That’s awesome! I’ll check that out and plan on getting a v6 to assure it has HT.

 

[Stux]

Use IPMI to upgrade the BIOS. You don't need a CPU then.

Course you need to buy an IPMI feature key to do that, but it is a way out if you get stuck with an old bios.

That or the cheapest nastiest v5 pentium you can find ;)

 

[kernalzero]

Course you need to buy an IPMI feature key to do that, but it is a way out if you get stuck with an old bios.

That or the cheapest nastiest v5 pentium you can find ;)

This thread seems to detail this some.
https://forums.freenas.org/index.php?threads/source-for-supermicro-sup-ipmi-license-key.40527/

I don't think this is the right product though becauset he x11ssm can support i5s. "Out of Band Firmware Management License-BIOS Flash /Setting - (Motherboards that use i3, i5 and i7 CPUs do not support OOB)"
http://www.wiredzone.com/supermicro-software-management-sft-oob-lic-10024441

Use IPMI to upgrade the BIOS. You don't need a CPU then.

It looks like you might have gotten the key for free. I saw this in another thread - did you just email supermicro support?

 

[Inxsible]

It looks like you might have gotten the key for free. I saw this in another thread - did you just email supermicro support?

Wasn't me!!

I upgraded my X9 board using the old method with FreeDOS. For this option, you'd need a compatible CPU though. My IPMI didn't even have the option to buy a key and update the BIOS.

Email SM and ask if they will send you a key for free.

 

[pro lamer]

x11ssm can support i5s

i3 not i5.

It may make a difference.

EDIT: I think Supermicro would write explicitly if the motherboard didn't support out-of-band BIOS update. At least they did it for some Xeon-W motherboards e.g. X11SRA-F EDIT2: Personally I would check the X11SSM manual to make sure.

 

[r0nski2000]

@op
Just an fyi, I bought an x11ssm-f-o(Amazon I think) last July(2017) and it came with bios 2.0a, if I remember correctly - regardless, it was fine with xeon v6 out of the box..

 

[yacenty]

Today I had bought second x11ssl - f.
The mb automatically updates BIOS from 2.0c to 2.1a - without any user works. No downloading file, no entering enywhere
After initial boot - the system restarts 3 times and You have bios - 2.1a. I have no special license on the supermicro.
Finally with new piece of mb - I managed to work with 4x 16gb Samsung memory so I have 64gb ram.

 

[kernalzero]

That's all great news. I've started ordering parts at this point so the fun is soon to come. Thanks everyone.

 

[kernalzero]

Final thoughts:

I completed my build which in the end was:

• SuperMicro X11SSM-F (motherboard was already on v2 so it took the v6 xeon without issue)
• Intel E3-1240 V6
• Crucial 16GB MTA18ASF2G72AZ-2G1A1 ECC x2 (32GB total)
  
• WD Red 4TB x6 (24TB total, RAID-Z2 14.04TB)
• Mirrored PNY 120GB SSDs x2 (I couldn't find cheaper, smaller drives to run OS)
• Fractal Node 804
• SeaSonic 650 (Gold)
• Eaton 5S700LCD UPS

The burn-in process was tedious but not terrible. It took a total of 155 hours, the majority of which was spent testing the hard drives. Initially I followed one of the tutorials here on the forums but I found the WD Data LifeGuard Diagnostics application on UBCD able to only test a single drive at a time. I ended up using the MediCat USB and I was able to test 6 drives simultaneously which decreased the testing time significantly. Each drive had 4 passes, so did the SSDs along with the DDR4. I'll admit the MediCat route was the lazy route but I frankly didn't want to learn how to use tmux for this sole task. CPU testing i just ran a script to peg out the CPU using a linux live of CentOS.

bash scripts:
peg cpu to 100% on all 8 cores:

Code:

for i in 1 2 3 4 5 6 7 8; do while : ; do : ; done & done

kill above script

Code:

for i in 1 2 3 4 5 6 7 8; do kill %$i; done

For those struggling with the mobo pin out with the Fractal Node 804

This is how @JWV solved it:
(1) PWR in pin 1 and 2 (with the white "PWR" text facing pin 3 and 4)
(2) PWR LED + in pin 15
(3) PWR LED - in pin 16
(4) HDD LED doesn't work (not with the + in pin 14 and the - in pin 13, nor with the + in pin 13 and the - in pin 14)

I've got FreeNAS 11.1U5 installed, I've managed to setup all the shares I want for the house, added the plex plugin (which I may switch to a jail because it seems to be easier to update) and I'm working on getting transmission up and running with PIA VPN. I'll also start reading more about getting a few VMs up and running too. I do wish I had gone ahead and built a FreeIPA or OpenLDAP box before this as I think I would have preferred the integration but I may eventually get to that. Thanks for all your help everyone who chimed in.

 

[Inxsible]

(4) HDD LED doesn't work (not with the + in pin 14 and the - in pin 13, nor with the + in pin 13 and the - in pin 14)

I have had the same issue with Node 304. The HDD LED just doesn't want to work. Although my motherboard (Tyan S5533) has a 9/10-pin front panel connector.

I was playing around with putting the HDD LED pins and the LED lit up once-- so I know the LED is not blown, but haven't been able to get it to work consistently. But then again, my box sits in a server closet, so I could care less if the LED works or not.

 

[kernalzero]

I have had the same issue with Node 304. The HDD LED just doesn't want to work. Although my motherboard (Tyan S5533) has a 9/10-pin front panel connector.

I was playing around with putting the HDD LED pins and the LED lit up once-- so I know the LED is not blown, but haven't been able to get it to work consistently. But then again, my box sits in a server closet, so I could care less if the LED works or not.

Yeah I lost interest in trying to get the light working. I know the drives are working and I can check the reporting tab to see more detail.