BUILD Should have done my homework (or) thoughts/issues on the SuperMicro MBD-X9SBAA-F-O

[pill128]

In all fairness to myself, I did spend quite a bit of time lurking/learning on the forums prior to ordering parts for my FreeNAS server, but I didn't exactly take everything i learned to heart. I started out with two requirements that I've learned, really shouldn't be starting points for a NAS: 1) small form factor (mini ITX - bought the LIAN LI PC-Q08B almost a year prior to the rest of the box) 2) as low power consumption as possible.

Now I did take some of what i learned to heard including buying a server grade board, a 'real' NIC, using ECC RAM and getting an excellent PSU. As the result of my requirements, my server board choices were very limited. Wanting a 'modern' Atom processor and IPMI, I was basically looking at 1 option on the market currently - the SuperMicro X9SBAA. Having seen just passing comments about this board on the forums, I went ahead and ordered the board and the remaining parts, even with some obvious limitations, 4x SATA ports, No Internal USB Header/port, USB 3.0 only, 1 xPCI-32 slot, 1 ECC SO-DIMM Slot - which leads me to the rest of the build:

SeaSonic SS-400FL2
8GB ECC Kingston SO-DIMM
3 x ST3000DM001 3TB
Rosewill 4+1 VIA USB 2.0 PCI Adapter Model RC-103

I wasn't looking for a massive amount of storage so I wasn't overly concerned about the limited and non-expandable SATA ports (at the time). I wasn't happy about not having an internal USB port will little ones at home and I understood that USB 3.0 was a non-starter win FreeBSD, so i ordered the USB 2.0 PCI card. I wasn't looking for anything extremely fast, so i skipped the warnings about Atoms being under-powered. Which brings me to my experiences and issues from the last few weeks:

• No matter what i tried, i could not get FreeNAS to load from the PCI USB card. Maybe this should have been obvious, but no where in the BIOS could i find the ability to boot from the card. Booting from the onboard USB 3.0 ports resulted in the well documented errors. The fix: i have a FreeNAS running on a 60 GB SSD i had laying around now chewing up 1 of 4 SATA ports - slightly more costly than a 4 GB Flash drive.
• IPMI - spent some extra dough to get this feature and for whatever reason does not seem to work after booting. I can use a remote keyboard to get into the BIOS, etc, but interaction with the console is not possible. I have flashed it to the newest firmware Supermicro has available and still no joy.
• The Atom processor - yep, it's slow...the GUI is slow. File Transfers seem slower than they should be. Plex does stream fine, but i have yet to push content to multiple TV's at the same time.

As I stated previously, i could find very few threads on the board (including one from a dev commenting on the impact of USB 3.0 and 'good FreeNAS candidate boards' like the X9SBAA) which is a big part of the reason i'm posting this thread.

If your considering this board for your NAS, do yourself a favor and buy something else unless you know exactly what you are doing and are cool with the severe limitations it presents.

If anyone is still reading at this point (yeah i know tl;dr), if there are any meaningful logs i could provide in relation to the IPMI issue, I would be more than happen to do so.

Thanks

 

[jgreco]

The BIOS probably has no clue about your PCI USB. Can't boot from what you cannot talk to.

I am also wondering if your IPMI issue relates to USB, as the IPMI keyboard/mouse are presented as USB to the OS, and FreeNAS has USB3 disabled.

So:

1) Try enabling USB3 on the FreeNAS system.

or

2) Try an earlier FreeNAS 9 that had USB3 enabled by default.

 

[pill128]

Thanks for the quick response jgreco! Good call on the enabling USB3 on the system - it just occurred to me that i had the same experience when i directly connect a USB keyboard to the box: I could get into the BIOS, but can't interact with the console. I'll do my own digging here, but by enabling USB3 on the FreeNAS system, this would be accomplished via a tunable?

Regarding the first issue...i'm assuming there is no way to make the BIOS 'aware' of the PCI USB? Despite all that went wrong with this build, I'm good with it for now and i'll revisit a more intelligent build in a year or two, so its not the end of the world if i need to keep rolling with the SSD.

 

[jgreco]

Sorry, on a phone and USB ain't my thing. All I csn really tell you is that USB3 support was causing problems and got turned off by default recently. If it doesn't crash or cause trouble when turned on for you, I'd guess that its disabling is the problem, and enabling is the IPMI fix - probably.

Booting from USB requires the BIOS to have basic driver support for USB. I don't know enough about USB add-on cards to comment on expected behaviours, but if it isn't showing it under PCIe config (especially option ROM) or boot order options or anywhere else, guessing not.

 

[Dusan]

The problem with X9SBAA is that it does not allow you to access the USB ports via USB2, see: https://bugs.freenas.org/issues/3273#note-3 and http://forums.freenas.org/threads/installations-problem.15429/
So, with the xhci (USB3) driver disabled all USB ports on that board are unavailable.
You can try this to enable the xhci driver:

1. Quickly hit Esc on the Welcome to FreeNAS boot prompt ([Esc]ape to loader prompt)
2. load xhci
3. boot

However, it is possible that the driver is not compatible with your USB controller anyway :(.

 

[pill128]

Thanks Dusan...will setting a tunable of xhci_load to YES work too? I was able to build the box by getting an address via DHCP so i haven't needed to use the console. I've got a file transfer that should complete in 20-30 mins and i can reboot the box.

 

[Dusan]

Yes, xhci_load=YES does the same.

 

[Satam]

The Atom processor - yep, it's slow...the GUI is slow. File Transfers seem slower than they should be. Plex does stream fine, but i have yet to push content to multiple TV's at the same time.

So maybe getting a Xeon E3 1230 v3 instead of an "Avoton" was the right choice for me after all...

But somehow I get the impression that performance-wise FreeBSD components aren't all optimized well for most newer CPUs. The geli benchmarks look really disappointing for example.

 

[warri]

So maybe getting a Xeon E3 1230 v3 instead of an "Avoton" was the right choice for me after all...

But somehow I get the impression that performance-wise FreeBSD components aren't all optimized well for most newer CPUs. The geli benchmarks look really disappointing for example.

If you look for example at the C2750 (8 core, max 64 GB RAM and AES-NI) instead of the S1260 (2 core, max 8 GB RAM) used here the picture might change a bit.

 

[pill128]

ahh interesting warri....getting a bit off topic but any idea if the proc in the x9sbaa is replaceable? looks like the C2750 and S1260 share the same socket... this is exactly the territory i didn't want to get into (part of the reason for the board to begin with)..

 

[pill128]

The problem with X9SBAA is that it does not allow you to access the USB ports via USB2, see: https://bugs.freenas.org/issues/3273#note-3 and http://forums.freenas.org/threads/installations-problem.15429/
So, with the xhci (USB3) driver disabled all USB ports on that board are unavailable.
You can try this to enable the xhci driver:

1. Quickly hit Esc on the Welcome to FreeNAS boot prompt ([Esc]ape to loader prompt)
2. load xhci
3. boot

However, it is possible that the driver is not compatible with your USB controller anyway :(.

Seems like the driver isn't compatible with the controller...Still no KB over IPMI

 

[Dusan]

getting a bit off topic but any idea if the proc in the x9sbaa is replaceable? looks like the C2750 and S1260 share the same socket...

Unfortunately not, FCBGA is not a socket but a surface mount packaging, meaning the CPU is directly soldered to the motherboard.

But somehow I get the impression that performance-wise FreeBSD components aren't all optimized well for most newer CPUs. The geli benchmarks look really disappointing for example.

The geli benchmark thread numbers are no longer valid. 9.2.0 contains several changes that enable drastic geli speedup.

 

[Satam]

If you look for example at the C2750 (8 core, max 64 GB RAM and AES-NI) instead of the S1260 (2 core, max 8 GB RAM) used here the picture might change a bit.

Oh, heh.

Hm, considering the very weak CPU and the issues with USB3.0, maybe you should return the board while you still can. The keyboard and usb boot issues will probably be fixed in a later FreeBSD release, but the poor performance of the S1260 will stay. Even the latest C25xx SoC-boards from Supermicro also come with USB2.0 ports, although the onboard header is already USB3.0 (no idea if it has USB2.0 fallback). So all your problems would be solved: console, usb stick boot and adequate performance.

 

[Mguilicutty]

If you decide to go another route and cannot return the Atom, it would probably make a nice PFsense or similar firewall. You could pop it into one of SM's 1U Atom cases, they are pretty cheap if I recall.

 

[Satam]

Yeah, very good idea with the firewall. It has two GbE ports. That board is definitely not bad in itself, just not right for the initially intended purpose.

 

[pill128]

Oh, heh.

Hm, considering the very weak CPU and the issues with USB3.0, maybe you should return the board while you still can. The keyboard and usb boot issues will probably be fixed in a later FreeBSD release, but the poor performance of the S1260 will stay. Even the latest C25xx SoC-boards from Supermicro also come with USB2.0 ports, although the onboard header is already USB3.0 (no idea if it has USB2.0 fallback). So all your problems would be solved: console, usb stick boot and adequate performance.

I imagine its too late to return the board, I have this stuff around since November and just got to building it recently :( I'll have to look into the C25xx stuff.

If you decide to go another route and cannot return the Atom, it would probably make a nice PFsense or similar firewall. You could pop it into one of SM's 1U Atom cases, they are pretty cheap if I recall.

Great call! My next project was to build a pfsense box and to straighten out my network with pseudo enterprise grade gear (ubiquiti AP and switch). I imagine this would work well for pf. I've gotten a bit spoiled working in an extremely large Cisco shop and am getting fed up with the "wireless/network isn't working" texts from my wife (the 6-8 AP's I've bought in the last few years doesn't help either).

What's funny about this whole debacle is I originally built up FreeNAS on an older Dell business desktop (shipped with Vista) to get an idea of the difficulty in building/configure it. Install couldn't have gone more smoothly and i was streaming movies off of it without a hiccup within two hours.

I'd really like to thank everyone for responding so quickly - honestly, i thought i was going to get yelled at for being a knucklehead...y'all are a good group of geeks!

 

[jgreco]

Waaaaallllllll... we can do that too.

KNUCKLEHEAD!

 

[Mguilicutty]

Off topic but good choice on the Ubiquiti gear as well. I use their APs for my clients that can't afford Cisco stuff as well as in my house. I've not had one die on me yet and I can manage them all from anywhere. I've got at least 15, maybe 20 installed in various locations. As for switches, 3750Gs can be found on Ebay fairly cheaply if you want to stick with Cisco and you do a bit of hunting around.

 

[pill128]

Off topic but good choice on the Ubiquiti gear as well. I use their APs for my clients that can't afford Cisco stuff as well as in my house. I've not had one die on me yet and I can manage them all from anywhere. I've got at least 15, maybe 20 installed in various locations. As for switches, 3750Gs can be found on Ebay fairly cheaply if you want to stick with Cisco and you do a bit of hunting around.

Good to hear from a 'real' person that Ubiquiti is as good as it looks/sounds. As for the 3750 at home - I look at their stuff day in and day out...I'd rather not come home and see the bay bridge arches. With the Ubiquiti AP's not being standard POE, I'm going to give either the 4 or 8 port switch a shot.

 

[Mguilicutty]

I've not tried the Ubiquiti switches yet. The APs all come with injectors so I've just been using those and I picked up a couple of spares just in case. It does suck that they are not standard POE, but I think the newer 802.11AC ones might be.

Be careful though, I've seen it before. Now you've got your shiny new NAS and a real firewall and it sounds like a new AP or two down the road. Next thing you know you'll be running vlans to keep the kids and their friends away from the important stuff and then your doing layer 3 switching. Next you'll end up with a 4500 down in the basement just so you can do netflow to see which of the little bastards is eating up all your bandwidth. Before you know it you end up filling 42 Us with stuff you never imagined.