Super Micro 4U Server Questions.

[Bozon]

I am thinking about purchasing a used Super Micro 4U Server for upgrading my very old and reliable FreeNas Server. I have a few questions, and I am pretty clueless about hardware, and what I need. The main thing that I use my FreeNas server for is backing up the 10+ Macs I use at home for business, development and personal use. The other application, I would like to run is a VM running PostgresQL, as I am a DBA in my main life, and would like to have a SQL database server at home.

My limitations: I can spend about 1000+ dollars total. I have limited time to hunt best deals, and assemble a server from scratch. I would trade money for convenience.

Here is a server I was thinking of buying:

Welcome to The Server Store - Used & Refurbished Servers at Great Prices

Is this a reputable vendor? Is there a better vendor, to use? Are the used 3TB SAS drives, worth the $48 dollars they are charging for them, or should I steer clear of them? How much memory should I have? Which CPU's should I choose given the options. Also, please include any other options or questions I should consider.

Thanks for your time and consideration.

 

[Bozon]

Maybe $48 Dollars is too much. https://www.ebay.com/itm/Refurbishe...e=STRK:MEBIDX:IT&_trksid=p2060353.m1438.l2649

I saw this listing on ebay for basically 26.60 per drive if you buy 4 or more.

 

[drinking12many]

Thats basically what I am using for me it works great. I run 2 full VMs on it and maybe 10-12 jails. I also run some docker containers on one of the VMs it works really well. I have maybe 12 drive bays full. I use several SSD for cache drives and my jails fully sit on small flash drives. Most of the time my CPU sits at less than 10% with dual 6 core E5-2630L V2 low power CPUs. The whole box usually is just shy of 300w all the time I have a 10Gb NIC in mine as well running ISCSI one port to each VM host running ESX.

 

[PhiloEpisteme]

Hi @Bozon. I've had great luck purchasing used chassis from Ebay. The things to look for are whether they come with power supplies, backplanes, and fans. Also double-check that they support the form factor of your board, most do. Also, avoid chassis which do not come with drive caddies, those things can be pricy to purchase after-the-fact.

Regarding used HDDs. I purchased a handful for use in a primary pool and for testing; they were terrible. Several shipped with SMART errors and others died soon after I got them. My recommendation is that if you're buying used drives insist on them running a long SMART test prior to sending it your way and ask that they then give you the output of smartctl -a for each device. Refurbished is a general word and can mean as little as the buy cleaned off the drive for you. What really matters is how many reallocated sectors the thing has, what the high temps have been, how many spin cycles has it gone through, how many power on hours it has, has it experienced and unrecoverable read errors, etc. With the drive you're looking at compare that price to the new price. If it seems way too good to be true, it probably is. Of course, it all comes down to what your risk tollerance is and how your pool is designed. If you're running 6-8 drive RAIDZ3 that is one thing compared to say a 12 drive RAIDZ1.

 

[Bozon]

Hi @Bozon. I've had great luck purchasing used chassis from Ebay. The things to look for are whether they come with power supplies, backplanes, and fans. Also double-check that they support the form factor of your board, most do. Also, avoid chassis which do not come with drive caddies, those things can be pricy to purchase after-the-fact.

Regarding used HDDs. I purchased a handful for use in a primary pool and for testing; they were terrible. Several shipped with SMART errors and others died soon after I got them. My recommendation is that if you're buying used drives insist on them running a long SMART test prior to sending it your way and ask that they then give you the output of smartctl -a for each device. Refurbished is a general word and can mean as little as the buy cleaned off the drive for you. What really matters is how many reallocated sectors the thing has, what the high temps have been, how many spin cycles has it gone through, how many power on hours it has, has it experienced and unrecoverable read errors, etc. With the drive you're looking at compare that price to the new price. If it seems way too good to be true, it probably is. Of course, it all comes down to what your risk tollerance is and how your pool is designed. If you're running 6-8 drive RAIDZ3 that is one thing compared to say a 12 drive RAIDZ1.

Thanks good advice, and somewhat deflating, I had hoped for something too good to be true. Maybe the Easter Bunny will bring me some new drives with my candy basket.

 

[PhiloEpisteme]

Haha yeah. I mean, it's a bit of a scale isn't it? Data with backups in a high redundancy setup is more resilient than a low redundancy setup with no backups. It also matters how available and familiar you are with doing drive replacement and how critical the data is. If you follow the 3-2-1 backup scheme you're in a much better position to take the chance than if you keep only 1 copy of your data. I'm sure there are gold mines of used drives out there.

 

[Bozon]

For the 9210-8i flashed to IT mode, how do I connect it to all of the drives. Do I use something like this? Intel RES2SV240 and sas to 4 sata cables or is there another better option.

 

[Bozon]

I literally had not heard of the 3-2-1 backup scheme. https://www.nakivo.com/blog/3-2-1-backup-rule-efficient-data-protection-strategy/ In case someone else hadn't heard of it.

 

[danb35]

Do I use something like this? Intel RES2SV240 and sas to 4 sata cables or is there another better option.

I'm pretty sure that server already has the appropriate SAS expander backplane to do this--you just need a SAS cable from the HBA to the backplane.

 

[drinking12many]

Yes any of the supermicros like that already have the SAS backplane usually its somewhere in the specs which it is as there are a few different versions of backplane that tend to be used. There are ones on ebay that will say they are for Freenas and already have the right type of HBA if you look as well.

 

[Bozon]

Yes any of the supermicros like that already have the SAS backplane usually its somewhere in the specs which it is as there are a few different versions of backplane that tend to be used. There are ones on ebay that will say they are for Freenas and already have the right type of HBA if you look as well.

Thanks. Even though, I have a minor in computer engineering, I can't get my eyes to not glaze over when reading hardware specifications. This is probably why I am a DBA. Thanks. This is especially true of Intel CPU specifications, which I know exist only to maximize their revenue from the same chip design by selling every different version based on the same silicon for the maximum amount of money they can get for it. Arrrgh.

 

[danb35]

Yes any of the supermicros like that already have the SAS backplane

They may have a SAS backplane, but they don't necessarily have a SAS expander backplane--there are lots of the -TQ systems out there that are pretty much a straight pass-through. Cabling on the 2U variants is just at the edge of tolerable; with the 4U ones it's a rats' nest. But the E16 in the model number indicates a SAS expander backplane.

 

[Bozon]

They may have a SAS backplane, but they don't necessarily have a SAS expander backplane--there are lots of the -TQ systems out there that are pretty much a straight pass-through. Cabling on the 2U variants is just at the edge of tolerable; with the 4U ones it's a rats' nest. But the E16 in the model number indicates a SAS expander backplane.

Yes, thanks it's actually in the description, but of course my eyes glazed over after "DUAL INTEL..."

• Processors: DUAL INTEL XEON PROCESSOR E5-2620 SIX CORE 15M CACHE 2.0GHZ
• Memory:16 Dimm Slots Available
• Hard Drives:24x 3.5" SAS/SATA Drive Bays Available
• Raid Controller:No Controller Installed (2x SAS Cables Included)
• Power Supply:2x PWS-1K21P-1R 1200W POWER SUPPLIES
• System Board:X9DRI-F
• Backplane:BPN-SAS2-846EL1

https://www.supermicro.com/manuals/other/BPN-SAS2-846EL.pdf

So the 2 included SAS cables connect from the LSI9210-8i to the backplane, of course flashed to IT mode.

 

[danb35]

So the 2 included SAS cables connect from the LSI9210-8i to the backplane

It really only needs one; I don't believe a second cable benefits you, though the manual (which you've linked) should address that in more detail.

 

[PhiloEpisteme]

You can use only 1 cable, but using two should give you more bandwidth between the backplane and your system. I asked a similar question to supermicro about that exact same backplane.

 

[Bozon]

SAS2-846EL1 backplanes have a single-port expander that accesses all hard drives and supports cascading. SAS2-846EL2 backplanes have dual-port expanders that access all the hard drives. These dual-port expanders support cascading, failover, and multipath.

It looks like you have to have more than 1 backplane to get a performance bump from the EL1. And the EL2 looks like a nicer version to have because it's model number is 1 higher, and also probably because it has dual-ports versus the single-port that the EL1 has. More being better in this case. I included the page in the manual that talks about this in more detail, with a picture and everything.

 

[drinking12many]

Doesnt hurt to have both cables hooked up to even the same card. I had a card that one SAS port died but the other ticked along until I ordered a new card.

 

[Bozon]

Doesnt hurt to have both cables hooked up to even the same card. I had a card that one SAS port died but the other ticked along until I ordered a new card.

Is that what the manual means by cascading

 

[dsrtusr88]

Curious here trying to figure out what HBA I need, if any. My mobo I plan on installing has a SlimSAS connector PCIe 4.0 4x that it says can take up to 4 SATA drives (1 x SlimSAS Slot Support SlimSAS NVMe device (supports PCIe 4.0 x4 mode and up to 4 SATA devices). Curious if there is a cable in existence that can just direct connect the moBo to the backplane with no HBA needed. Backplane is the SAS2-EL1