So, you’ve decided to buy a Supermicro X11 Xeon E3 v5/6 board...

[religiouslyconfused]

Nice, would love to have a board like that but would settle for the X11SSH-F.

 

[xhoy]

Nice, would love to have a board like that but would settle for the X11SSH-F.

Why ?? it misses the LSI controller and the 10GBit nic's, I would really love this board (CTF) with a full ATX (and thus more expansion slots) fur the future expansion.

 

[Ericloewe]

Why ?? it misses the LSI controller and the 10GBit nic's, I would really love this board (CTF) with a full ATX (and thus more expansion slots) fur the future expansion.

There's only so much PCI-e 3.0 connectivity to go around, so such a board would not be as useful as a Xeon E5 board. A single E5 provides 40 + 8 (or so, don't remember the PCH layout) lanes instead of 16 + 8 (that get multiplexed over a crapton of things - SATA, PCH PCI-e connectivity, USB, ...)

 

[xhoy]

So the X11SSH-CTF just arrived, starting test next week. I'll keep the updates comming in. Any specifics things somebody likes me to test?

 

[Ericloewe]

Just tell us how things go with the 10Gb NIC and the SAS3 controller.

 

[xhoy]

Ok so we started with randomly installing W10, the X550 do not work by default on windows as some other stuff but that was solved rather quickly. Yesterday we started a memtest and it ran for 16 hours and did found error.... Just restarted the test with half of the memory and it did find errors, this time in 6 minutes :) Keep you posted when i learn more :)

 

[xhoy]

One thing i did noticed (and the reason i haven't installed freenas yet) is that the board uses mini-sas HD connectors (SFF-8643), we in tent to use it with standard sata disks or at least standard sata cables. When I looked on line and found only extremely expensive once... (€99,-!) Then I looked at the website of SM and found that they have a mini-sas to sata cabels.

The cables only cost €9 apiece!
CBL-SAST-0556 => internal mini SAS HD to 4 SATA, 90/90/75/75cm w/75cm SB, S. 30AWG SAS/SATA Internal miniSAS HD to 4 SATA
CBL-SAST-0631 => cross-over mini SAS HD to 4 SATA w/SB, S. 55cm. 30AWG SAS/SATA Internal miniSAS HD to 4 SATA Cross-over 55 cm
CBL-SAST-0591 => cross-over mini SAS HD to 4 SATA w/SB, S. 75cm. 30AWG SAS/SATA Internal miniSAS HD to 4 SATA Cross-over 75 cm

Just seconds ago the single stick did fail again so i think we have a broken memory stick....

 

[xhoy]

So, there is a broken memory stick, so we are stuck with 48GB instead of the 64GB.

For your interest's here is our config:

MB: Supermicro X11SSH-CTF
CPU: Intel Xeon E3-1245 V5 (3.5ghz)
Memory: 4x Kingston 16GB ECC (KVR21E15D8/16)
HDD: to be determined likely 8x4TB something :)


Just installed FreeNAS-9.10-STABLE-201604111739 (896cc83) and the 10GBit nics work like charm :) Now doing some speed tests to look at the performance, but i have a no other system with 10GBit nic's.....

 

[Dudde]

I'll have an similar setup, X11SSH-CTF , 64GB ECC, Xeon E3 etc
I got everything up and running except that FreeNAS complains about the LSI 3008 firmware version.
I flashed it to IT mode with the latest driver from Supermicros ftp site, 3008_FW_PH10.00.03.00.zip
But since FreeNAS (FreeNAS-9.10-STABLE-201604111739) is using the PH9 drivers im getting the following message

Code:

WARNING: Firmware version 10 does not match driver version 9 for /dev/mpr0. Please flash controller to P9 IT firmware.

Some guys told me i could download the PH9 firmware from Avagos website, anyone tried this and got it working?
Or how did you solve the firmware version issue?

Except for this problem i've tried running some speed test with the 10GB/nic between my ESXI machine and this one, the results is pure stock results.
I haven't done anything more then installed the latest FreeNAS 9.10 version, iperf gives 9,45GB/s between the machines using simple cat 6 cable.

 

[RichTJ99]

Any updates? Was it an issue to install Freenas 9.10 on the USBs?

 

[Ericloewe]

Any updates? Was it an issue to install Freenas 9.10 on the USBs?

FreeNAS 9.10 installs just fine on USB drives, on Skylake systems.

 

[Dudde]

I had problems with 9.3 but 9.10 worked fine.

 

[Ericloewe]

XeonForo seems to have some trouble with certain characters.

The X10 thread was correctly rendered as "So you've decided..." in the title, even though the input box displays the apostrophe as a straight ASCII-style straight apostrophe. By inputting an apostrophe directly from the keyboard, I get shown an ASCII-style apostrophe, but the output is this weird, short apostrophe. So I grabbed the apostrophe from the old title, which the text box renders somewhat like an acute diacritic (acento agudo and variants, for Romance language speakers). Weird.

Anyway, OCD is satisfied and both threads have identical titles, minus the obvious family name difference.

 

[RichTJ99]

Is it possible to run memtest from a USB stick on these boards? I went for a new X11 also - bought the wrong memory but the 64gb replacement will be on hand Monday :) I would figure 5 or so days for 64gb on memtest.

 

[BigDave]

Finally took the plunge today and ordered a new x11ssm-f with a stick of samsung
m391a2k43bb1-cpb and an Intel G4400 cpu. :D new toys!

 

[Dice]

Is it possible to run memtest from a USB stick on these boards? I went for a new X11 also - bought the wrong memory but the 64gb replacement will be on hand Monday :) I would figure 5 or so days for 64gb on memtest.

yes.
Use a proper guide to get the UBCD sorted. I followed this.

 

[GiuseppeChillemi]

Finally took the plunge today and ordered a new x11ssm-f with a stick of samsung
m391a2k43bb1-cpb and an Intel G4400 cpu. :D new toys!

Could normal (non-ECC) DIMM be used ?

 

[Ericloewe]

Could normal (non-ECC) DIMM be used ?

Uncertain. It's a bad idea anyway.

 

[BigDave]

Could normal (non-ECC) DIMM be used ?

Not with my data :p

 

[Dudde]

Im having a problem with my LSI 3008 controller on my X11SSH-CTF board. After flashing it to PH9 to match the drivers in FreeNAS it now takes 3-4 minutes to pass the controller bios. It will eventually boot normal but it freezes for 3-4 minutes first.
I didnt have this problem with PH10 firmware. Any news on when FreeNAS will implement this driver?