So, you’ve decided to buy a Supermicro X11 Xeon E3 v5/6 board...

So, you’ve decided to buy a Supermicro X11 Xeon E3 v5/6 board...

Status
Not open for further replies.

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Note: This thread is now the discussion thread for this Resource, the current version of which can be found at this link, in the Resources section.

The old version is archived below.

In case someone still doesn't get the reference, click here.

Now that Xeon E3 v5 is here, motherboard manufacturers have finally released their server lineups. Much like the Supermicro X10 guide, the purpose of this guide is to clarify the differences between the various options offered by Supermicro. Workstation boards and models designed for legacy applications (PCI and/or PCI-X slots) and boards with proprietary sizes are not considered, though some workstation models may be of interest, should a good deal be available.

The full comparison is presented below, for prospective purchasers and for troubleshooting. If you are familiar with the Supermicro X10 lineup, a shorter cheat sheet is available in this thread’s second post.

Baseline features
Supermicro opted for a baseline closer to the X10SLL+-F/X10SLM+-F/X10SLH-F models for the X11 server motherboard lineup:

· Standard four DIMM slots, for up to four DDR4 16GB UDIMMs (64GB total RAM)
· IPMI provided by an ASpeed AST2400 BMC, with dedicated IPMI LAN port
· microATX form factor
· Dual Intel i210-AT GbE adapters
· Leftover PCI-e connectivity exposed in standard slots
· 5x USB 3.0 ports and 6x USB 2.0 ports
· C232 boards provide 6x SATA 6Gb/s ports
· C236 boards provide 8x SATA 6Gb/s ports and additional PCI-e connectivity

X11SSL-F
The X11SSL-F employs a C232 PCH and is therefore limited to 6 SATA ports, like Haswell/Broadwell systems.

PCI-e 3.0 connectivity is as follows:

· One x16 slot (x8 electrical) from the CPU
· One x8 slot from the CPU
· One x8 slot (x4 electrical) from the PCH

A cost-reduced version without IPMI, X11SSL (no –F) is also available.

X11SSL-CF and X11SLL-nF
Despite the name, these seem to not use the X11SSL-F layout. Feature-wise, they are similar to the X11SLL-F, though.

· X11SSL-CF routes 8x PCI-e 3.0 lanes to an LSI SAS 3008 controller, partially compensated by an additional x1 PCI-e slot
· X11SSL-nF routes a total of 8x PCI-e 3.0 lanes to two U.2 connectors, allowing for two PCI-e SSDs to be attached via cabling

X11SSM-F
Unlike previous Haswell/Broadwell PCH families (Lynx Point), the Sunrise Point family has only two versions (C232 and C236). For this reason, the X11SSM-F is positioned differently than the X10SLM+-F.

By employing the C236 PCH, the X11SSM-F provides 8 SATA ports, allowing for a typical 6-drive pool plus two ancillary drives, like SATA DOMs or SSDs for booting, L2ARC, mid-range SLOG devices, or simply larger pools.

PCI-e 3.0 connectivity is as follows:

· One x16 slot (x8 electrical) from the CPU
· One x8 slot from the CPU
· Two x8 slots (x4 electrical) from the PCH

A cost-reduced version without IPMI, X11SSM (no –F) is also available.

The X11SSM-F seems to be based on the same layout of the X11SSL-F.

X11SSH-F
The X11SSH-F is the high-end Xeon E3 v5 board from Supermicro. In practice, feature-wise, it’s very similar to the X11SSM-F. In fact, the only difference is that one PCI-e 3.0 x8 (x4 electrical) slot is exchanged for an M.2 connector (2x PCI-e 3.0).

For the sake of consistency, PCI-e 3.0 connectivity is as follows:

· One x16 slot (x8 electrical) from the CPU
· One x8 slot from the CPU
· One x8 slot (x4 electrical) from the PCH


X11SSH-LN4F
The X11SSH-LN4F is very similar to the X11SSH-F. The only difference is the addition of two Intel i210-AT GbE controllers (total of 4 i210s), using the two PCI-e lanes leftover from the removal of a slot.

X11SSH-TF
The X11SSH-TF uses a different layout than the X11SSH-F (it is similar to the X11SLL-C/F-nF layout, but different). Instead of Intel i210s, it uses the brand-new Intel X550 10GbE controller for two 10GBase-T ports.

The M.2

PCI-e 3.0 connectivity is as follows:

· One x8 slot from the CPU
· One x4 slot (x2 electrical) from the PCH (presumably)

X11SSH-CTF
The X11SSH-CTF is similar to the X11SSH-TF, but adds an LSI SAS 3008 SAS3 controller.

This makes the X11SSH-CTF a very interesting board, since its connectivity is as follows:

· Dual 10GbE from an Intel X550 10GbE controller (no driver support yet)
· 8 SATA 6Gb/s ports
· 8 SAS3 ports (LSI SAS 3008, not yet guaranteed to work well with FreeNAS)
· 1 M.2 slot backed by PCI-e 3.0 x4

Driver support should now be in place for all hardware on these boards, with FreeNAS 9.10.
There have been no noteworthy complaints about LSI SAS3 controllers using P9 firmware/drivers and the Intel driver for 10GbE has been updated with support for the X550 controller.

-O vs -B model numbers

Many people get confused with these model numbers when they can only find an X10SL7-F-O, instead of an X10SL7-F, or an X10SLM+-F-B instead of an X10SLM+-F.
All model numbers are exactly as printed in their respective sections. For real products, they get either an -O or a -B appended to them, for retail or bulk packaging, respectively.
An example:
A Supermicro X10SL7-F-O is a retail-packaged X10SL7-F motherboard.
A Supermicro X10SL7-F-B is a bulk-packaged X10SL7-F motherboard.


What's the difference between them? With retail packaging, you get a box, with proper packaging materials. With bulk packaging, you get something along the lines of an antistatic bag, a bit of protective foam and some cardboard to wrap it all up.


Edit 2016-04-25: Updated driver support notice for the X11SSH-(C)TF
Edit 2016-05-07: Added -B vs -O section, copied from the X10 thread. Should've been here all along. Fixed title apostrophe to proper serifed version, instead of mysterious apostrophe-like symbol.

 
Last edited:

DaveFL

Explorer
Joined
Dec 4, 2014
Messages
68
I like these for a workstation. If you plan to run bare metal and will never need more than 64GB they are great.

As you mention. The support for the older hardware is better (not sure if his applies to OmniOS as well, it might not have issues).

I still like Xeon D as it has more cores and allows for more memory.

I'll probably pick one up for a Workstation - X11SSZ.
 

nidur

Dabbler
Joined
Feb 5, 2014
Messages
27
The X11SSL-CF would be my choice for a Fileserver today, but the X11SSH-CTF will hopefully get full support in the near future then it would be a definite winner.

For a workstation I would personally go for the X11SAE-M
M.2 (PCIe x4), PCIe x16 slot for large GPU, 2x USB 3,1 in M-ATX size.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
We got a guy with a successful setup on the X11SSH. I'm going to have him make a forum post to collect thoughts/experiences on it.
 

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
Yes, one user in the US has a successful X11SSH-F setup and he uses an SSD though for his boot drive, as Skylake only supports XHCI and FreeNAS has that disabled in 9.3. Version 10 should support this when it comes out.
 

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
@Raiz has successfully ran a X11SSH-F setup and it seems to work for him. Skylake C232 and 236 board should use the Intel i210 ethernet chipset, so that should be supported in freenas just fine.
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,410
Are there any other 'negative reports' on X11 boards as of now?
Other than what I know - the XHCI, the lack of support for boards with SAS3008, 10GbE not yet 'there'?

Cheers /
 
Last edited:

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
Are there any other 'negative reports' on X11 boards as of now?
Other than what I know - the XHCI, the lack of support for boards with SAS3008, 10GbE not yet 'there'?

Cheers / Dice
No issues are currently known besides the obvious ones:
  • USB only works with the XHCI driver
    • Some people have reported that IPMI HIDs do not work either, but I don't think it's a real issue. Others have used IPMI successfully. Besides, why would the BMC provide HIDs using the PCH USB ports? It would almost certainly present the OS with PS/2 or USB HIDs from a PCI-e controller - in the latter case, it would certainly be an EHCI controller.
  • The SAS3008 controller used in some boards is unproven and potentially unstable.
  • The Intel X550 10GbE controller used in some boards (at least some of them without any 1GbE controllers) is not yet supported in FreeBSD/FreeNAS. This should change "soon" and it'll be a heck of a controller. It'll also support 2.5GbE and 5GbE, when those start appearing.
 

RchGrav

Dabbler
Joined
Feb 21, 2014
Messages
36
The Intel X550 10GbE controller used in some boards (at least some of them without any 1GbE controllers) is not yet supported in FreeBSD/FreeNAS. This should change "soon" and it'll be a heck of a controller. It'll also support 2.5GbE and 5GbE, when those start appearing.

This can't happen soon enough...
 

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
I believe FreeBSD is getting X550 support as there are some drivers that appear to be compatible with certain versions of that chip. I think it will definitely happen for FreeNAS 10 which is coming out sometime this year.
 

Rand

Guru
Joined
Dec 30, 2013
Messages
906
By employing the C236 PCH, the X11SSM-F provides 8 SATA ports, allowing for a typical 6-drive pool plus two ancillary drives, like SATA DOMs or SSDs for booting, L2ARC, mid-range SLOG devices, or simply larger pools.
Does that mean that utilization of 2 DOM's will reduce the number of usable SATA ports to 6 ? Or are these on top of the 8 ports for a C236?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
C232, like older PCHs, provides 6 SATA ports.
C236 provides 8 SATA ports.

The DOM ports are just two of the aforementioned SATA ports, with provisions to directly power Supermicro DOMs.
 

Rand

Guru
Joined
Dec 30, 2013
Messages
906
Bummer :/
Does anyone know whether (when?) support for the x550's will make it into FreeNas 9.x?
 

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
Not sure, but I am sure you could get the X550 FreeBSD drivers and try to compile them for 9.x.
 

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
Also, on Intel website I can only find X540 drivers for FreeBSD, but FreeNAS 10 is not that far off and I am sure Intel is working on a driver or a driver already exists.

Plus, that Supermicro board (X11SSH-TF/CTF) is not even out yet, so you will have to wait anyways and it might not come out until after FreeNAS 10. Then you will need to wait for a driver or Intel will have a driver for FreeBSD that you can use.
 
Last edited:

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
I don't think that would be a good server board as it contains unnecessary features such as display port and Realtek audio which is unnecessary for a Nas server. Would make a good workstation board. Plus it only has 4 sata ports from what I have read.
 

religiouslyconfused

Contributor
Joined
Dec 14, 2015
Messages
184
Plus x550 chip is not compatible with freenas yet and I would recommend x11ssh-tf if you want a 10gbe board but nothing wrong with the x11ssh-f or x11ssm-f.
 

KJaneway

Dabbler
Joined
Jan 15, 2016
Messages
12
It was just my idea because the X11SSZ-TLN4F is easily available in contrast to the x11ssh-tf and cheaper. I want definitely a 10GBit NIC

But you've made a good point with the audio and displayport .
 
Status
Not open for further replies.
Top