Supermicro X9 whitebox

[koolmon10]

Hi all,

Currently I run FreeNAS 11.3 baremetal on a Dell R510. Specs:

Dual Xeon X5650
32GB ECC DDR3
8x 2TB drives (2x RAIDZ1 vdevs, striped to 1 pool)
Dell PERC H200 flashed
2x 16GB flash drives for boot

I also have a Dell R710 which is my VM host (dual X5560, 64GB, Windows Server 2016).

I am planning to replace both of these machines with a single machine running ESXi and virtualize FreeNAS. I have already read through the forums posts regarding virtualizing FreeNAS extensively and feel like this setup will handle my needs well.

Rosewill RSV-L4500
Supermicro X9DRD-7LN4F-JBOD (forum suggestions pushed me toward X9/X10 for virtualization stability)
Dual Xeon E5-2630L v2
64GB RAM from R710 (having this already largely influenced my decision for CPU and board)
Dell H200 from R510

I have not found a good power supply yet, suggestions are welcome. I plan to dedicate 32GB RAM to FreeNAS, PCI passthrough on the HBA, and at some point in the future get 2 small SSDs to replace the flash drives as boot devices, using the onboard SAS controller with PCI passthrough as well.

Any additional suggestions?

 

[troybs1d]

If you have a SuperMicro motherboard I'd suggest going with a SuperMicro chassis. I actually came across an almost perfect system based on what you're looking & it's being sold by UnixSurplus too.

SuperMicro 4U CSE-846 24 Bay SAS2 BP X9DRi-F/2x W/ 2x E5-2620 32GB IT MODE

I'm not sure if you can do it with this chassis but for the one server I built/manage at my job the CSE-846 model I picked out can support a 3.5" drive internally (between the Power Supply cage & the fan shroud/motherboard. I put in a 3.5" -> 2x 2.5" SATA Hotswap module & connected that to the SATA port on the motherboard for ESXi to use. You can also get an Oracle/Sun F80 (800GB of 4x 200GB SSDs) for cheap too.

The only "issue" I see with the listing above is that it comes with the "1R" series of PSUs which are pretty loud. I'm not 100% sure if you can swap them out for the "SQ" series of PSUs. I would contact UnixSurplus to ask as they would definitely know the answer & would probably have them in-stock too.

 

[koolmon10]

That actually looks like a good system, thanks for the tip. The seller isn't UnixServers though...?

I noticed one of your machines has the same CPUs I picked out, do you have any data on the power savings over a non-L CPU?

 

[troybs1d]

You are correct on the seller - I guess I was looking at a different listing. Very sorry about that.

I recently purchased 2x E5-2630Ls for I believe $25 total shipped as I needed 2x CPUs for my HP ML350P Gen8 to get all the PCIe slots active. My HP unit needs different retention mechanisms for the E5-26xx v2 series since the IvyBridge PCB substrate is 1-2mm thicker (basically HP engineered the system too well) but for straight-forward SuperMicro system the v2 CPUs should be just fine. After doing some research Intel never made a L series for the E5-2620 v2 but they did make a E5-2628L v2 & a E5-2630L v2 with clock speeds (and original MSRP) being the difference between them.

The TDP of the E5-2620 (v1) is 95W each & the E5-2630L v2 is 60W each. Those are under full load conditions so probably a difference of 70W or so under full load.

The average price of the E5-2630L v2 on eBay are around $30-$40 each & $65 for the occasional "paired" listing. It seems the E5-2630L v2 were implemented more than the E5-2628L v2 thus the used market on them is a bit more competitively priced to move this gear. Also the E5-26xx v2 CPUs are on the edge of being old enough the prices on them drops like a sheer cliff. I came across this listing with a pair of E5-2630L (v1) that are only $12 w/ Free S&H. The v2 CPUs will net you a bit higher clock speed along with a relatively marginal performance increase for the slightly newer architecture. The E5-26xx v3 series are LGA 2011-3 so v3 & newer aren't compatible.

You can always get started with the CPUs it comes with, buy the CPUs later & whenever you have to shut the machine down for maintenance or hardware changes you could swap out the CPUs then.

 

[koolmon10]

Yeah, I actually had already purchased the CPUs so I'm hoping the power saving will be worthwhile. I got them for $24/ea.

Regarding the PSUs, do you know how they compare to an R510 or R710? The fans in my servers are fairly loud, but I have them in a room where I don't really hear them. If they were any louder though it might be a problem.

 

[troybs1d]

I'm not familiar with the R510/R710 but I am familiar with the fans in the 1R suffix models of the SuperMicro PSUs & they are loud as a jet. SuperMicro also makes PSUs that end in SQ (Super Quiet) which from what I read are very home/small office friendly when it comes to noise levels.

The older SuperMicro 846 chassis' PDU aren't compatible with the newer SQ model PSUs but I'm pretty sure the unit above & newer SuperMicro chassis are compatible with them.