SOLVED SSD's not found on X10SL7-F after BIOS update?

[grobbarg]

Hi all,

Recently (now seemingly foolishly) I updated the BIOS on my X10SL7-F (from 2.2 - 3.0), I immediately noticed some of my VM's were not starting and data stores were missing. FreeNAS was OK.
I tracked this down to one of the SSD's not being found in BIOS, one was on port 0 and the other on port 1. Port 0 was showing empty in BIOS but port 1 was OK. Hooking both SSD's to the SATA 2.0 ports was OK.

After contacting Supermicro, we went through different tests which included another BIOS update, an IPMI update and swapping a non SSD drive to port 0 (this worked). Through these tests, port 1 started to act randomly and sometime shows up, sometimes not with every restart.

Supermicro eventually concluded there was a hardware fault, and would have to RMA the motherboard. Before I go through the hassle of doing so, I wondered if any one else has experienced this? bearing in mind all was well prior to the BIOS update. My workaround for the meantime is to use ports 4 & 5.

 

[Spearfoot]

FWIW, I too am running FreeNAS on a X10SL7-F board with version 3.0 of the BIOS and I do not have problems with either of the SSDs located on SATA ports 0 and 1. I use these as VMware datastores. So it seems reasonable to assume that you do indeed have a hardware problem.

 

[grobbarg]

Thanks Spearfoot,

Can I ask what procedure you used to update your BIOS?, i.e. via IPMI or DOS boot USB with the JPME1&2 jumpers switched?

Initially I updated via IPMI as I did for 2.2.
The ports 0 and 1 still work with non SSD drives, so something must have gone tit$ up with recognising SSD's during the update.
Oddly, IPMI reports 2 different BIOS versions, on the system main page it reports 2.2 (as we downgraded), however the hardware information page reports 3.0a under BIOS (3.0a being a pre release and tested on my motherboard). Just not relishing the thought of the RMA procedure!

 

[Spearfoot]

Can I ask what procedure you used to update your BIOS?, i.e. via IPMI or DOS boot USB with the JPME1&2 jumpers switched?

@grobbarg, I used IPMI and didn't tinker with any mobo jumpers.

Hmmm... you installed a pre-release version? I'm sure you already tried it... but I wonder if dropping back to 3.0 would help?

Did you try tinkering with any of the BIOS 'Advanced' SATA settings? Just FYI, here are my settings:



Like you, I would dread having to go through the RMA process and would be willing to try just about anything to avoid it!

 

[grobbarg]

I know... pre-release version, but all done under the direction of Supermicro's tech support. I already went back to v2.2 but no joy. A few more things to try tomorrow, such as other SSD's, and some straws to clutch at....

My SATA settings are as yours, thanks for that. I did try changing SATA device type to SSD though.

 

[grobbarg]

Update: I tried an old Crucial CT128M225 SSD in port 0 with no problems, every restart the Crucial is recognised in BIOS (v3.0 currently)... I stuck the Samsung Evo back in just in case, but nothing.
I'm waiting for Supermicro's thoughts on this. If it was faulty MOBO hardware, wouldn't it be faulty for all SSD's or HDD's? Confused.

 

[Ericloewe]

Could be an odd edge case. I'd also check for newer firmware from Samsung.

 

[Spearfoot]

I'd also check for newer firmware from Samsung.

Good idea, @Ericloewe - there's an 840 EVO firmware update available from Samsung dated April of 2015:

http://www.samsung.com/global/business/semiconductor/minisite/SSD/us/html/support/downloads.html

 

[grobbarg]

Thanks for the input guys, I truly thought we were onto something there. However, an apology is due, as I wrongly stated in my sig. the SSD's as being 840 Evo's when in fact they are '850' Evo's.
Anyway, after transferring the 500GB 850 Evo to my workstation and running Samsung Magician 4.6 (it has to be 4.6, as previous versions won't recognise an update is due), it reported the 850 already had the latest firmware (I also can't find any 850 updates on Samsung's downloads page). However, as there was already an '840' Evo in the workstation, it was prompted for an update. So I did the firmware update (now EXT0DB6Q) on that 840 Evo and transferred it to the server to test. The excitment was building at this point... unfortunately after booting into BIOS it too was not recognised.

And BTW, whilst rummaging around the workstation I remembered I had another Crucial SSD, (CT256MX100SSD1), so just for fun, I tried that in the server port 0, it was recognised immediately. Additionally throughout these tests, I'd transferred the 120GB 850 Evo back to port 1, it was recognised each time.

So, in summary:
840 Evo with firmware update and both 850 Evo's not recognised on port 0.
120GB 850 Evo OK on port 1 but not the 500GB 850 Evo.
Both Crucial SSD's tested were OK on port 0.
All SSD's OK on SATA 2.0 ports

It does initially 'seem' to point to a port 0 problem, but if port 0 was truly knackered, why do the Crucials work? and if Evo's don't work, why does the 120GB Evo work on port 1? Just getting myself more confused!

 

[Ericloewe]

Could be a very specific controller defect. I imagine specific AHCI functions are done by hardware dedicated to each port. It could be the Samsungs use an AHCI command that the Curcials don't.

 

[Spearfoot]

Very troublesome... at this point I believe your best option is to exchange the motherboard.

 

[grobbarg]

Solved!! - after advising Supermicro tech support of the Crucial SSD's working in port 0, and the Evo's not, they got back to me saying it is known for Evo's to have backplane issues in general.
With this in mind I tested the SSD out of the hot swap bay, with a SATA cable direct to port 0 - it was recognised immediately... what we cannot figure out is how it took a BIOS update to trigger all this, pure coincidence perhaps. Anyway the SSD is now residing in another bay and fully functional. Such a simple solution, it never even occurred to me to check this...
I hope this can help others avoid the pain I've been through should they experience such issues!

 

[Spearfoot]

With this in mind I tested the SSD out of the hot swap bay, with a SATA cable direct to port 0 - it was recognised immediately...

So the 850 EVO's don't work when you use a hot-swap bay, but work fine when connected directly.

What brand of hot-swap bay were you using, if I may ask?

 

[grobbarg]

Well, not as straightforward as that! - they do work and did work flawlessly up until that BIOS update inexplicably highlighted the issue with port 0.
However, port 0 does not now seem to be the issue. Perhaps the backplanes for the Evo's are slightly thinner than other SSD's, if they are known to have issues as Supermicro suggested.

Supermicro pointed me in the right direction eventually, and when tested directly with a cable and in another bay, the Evo was recognised immediately. Perhaps the Hot swap bay itself has a dodgy PCB or connector at bay 1, but the Evo's always worked in the SATA 2.0 ports, and the Crucials and HDD's always worked in bay 1 to port 0.
The 500GB Evo is now in bay 5 of the same hot swap drive bay, and an HDD is in bay 1, both working fine. As happy as I am with this ATM and not having to RMA, I am still baffled with the following:

1. Evo's were fine up until the BIOS update, which was done remotely via IPMI - i.e. no tinkering with the chassis or enclosure as it is hidden away.
2. Throughout the testing, both Evo's worked on SATA 2.0 ports.
3. Throughout the testing with both Evo's and Crucials, the power light and activity LED's were always active on bay 1.
4. HDD's and Crucials always worked on bay 1 to port 0.

I'm sure there is more that baffled me, but those are the main points. I've got two of those hot swap bays which are:

https://www.servercase.co.uk/shop/c...ata-hot-swap-into-3-x-525-drive-bays-sc-h500/

 

[zoon01]

New firmware/bios versions now official released i saw today!

Bios
X10SL7-F R 3.0a x10sl75_C17.ZIP

IPMI
X10SL7-F R 3.27 REDFISH_X10_327.ZIP

I have no changelog

 

[grobbarg]

Thanks, for the heads up zoon01, will have to test those some time, although since I swapped the bays for the SSD, the server hasn't missed a beat (apart from one power outage) which reminds me, I should really get a UPS...