SOLVED Help diagnose possible X10SL7 motherboard failure?

[Linkman]

Build as follows (also in sig):
-- Supermicro X10SL7
-- Xeon E3-1241 v3
-- 32 GB Crucial ECC RAM
-- CoolerMaster case / PC Power & Cooling 500 W power supply
-- 60 GB boot SSD
-- 2x 250 GB Samsung SSDs mirrored
-- 2x (4x 3TB WD HDDS RAIDZ2) (total 8 drives, two VDEVS, one pool)
-- FreeNAS 11-Stable (current release)

Motherboard is two months shy of three years old, so still within the Supermicro warranty period, but I am not familiar enough to know what that covers yet, if needed. Been running flawlessly for that time period.

End of last week noticed hadn't received the daily status email, then discovered the Web UI was not accessible, and my router status page listed the FreeNAS box IP as "Off" Could not access the server via network over either of the two Ethernet ports, but IPMI was working fine. Rebooted, and both igb0 and igb1 were listed as no carrier. Swapped cable from one port to the other, no change; swapped ports on my switch, no change. Rebooted into at least two older FreeNAS boot environments to see if some update had hosed it, no change. Assumed motherboard's network hardware had died. Amazon Primed an Intel 210 based network card (StarTech brand, i210 chipset) and had the same results, except now igb0, igb1, and igb2 were listed as no carrier. New cable to another switch, and the add-in network card began to work. Duh! Was it a cable or switch port!?

Rebooted, got the WebUI, and... now none of the eight HDDs are detected! They do not show under Disks in the Web UI, and when booting there are no console messages indicating they are present either (I do see the boot SSD, and the two Samsung SSDs listed on the console). The eight HDDs are all on the LSI SAS controller. The SSD pool is imported with no error.

My next step is to pull the network add-in card, see if the motherboard network ports work on same cable and switch port that the add-in card was working on, and then see if the HDDs reappear after the add-in card is removed -- could there be some sort of PCIe conflict?

Question - what's my next trouble shooting step before seeing what the Supermicro warranty is, and contacting their support? Re-seat everything, try another power supply, anything?

Thanks!

 

[Chris Moore]

Supermicro makes nice gear, but I woulnd't count on the warranty. I have dealt with them before and it has always been a pain. here is why:

Code:

This limited warranty applies only to the original purchasers of Supermicro's products who are direct customers or distributors of Supermicro

See where it says direct customers or distributors, if you didn't buy it directly from Supermicro and you are not a distributor, you don't qualify for the warranty. You have to go back to the company that sold it to you and hope that they will facilitate the warranty service with Supermicro.
I stopped spending the premium to buy their equipment new for home use because of all the trouble I had with that. I buy and use Supermicro gear new for work, but I go through a company that says they will handle the warranty service for me. At home, I save the money, buy it used, and don't worry about the warranty.

 

[Chris Moore]

what's my next trouble shooting step

Did you boot into the BIOS/UEFI config to see what hardware is showing up in there? You should be able to get some information about what devices are working through that.

 

[Linkman]

Did you boot into the BIOS/UEFI config to see what hardware is showing up in there? You should be able to get some information about what devices are working through that.

No, I did to check the NICs, but not after the HDDs went missing. I will add that to my to-do list for troubleshooting, hopefully tonight.

 

[Linkman]

Solved. Apparently a bad switch port.
-- Using the original CAT5 cable, but putting it on my second switch re-ignited the interface, DHCP worked, and I could access the server again.
-- Removed the add-in Ethernet card, and the HDDs on the LSI controller re-appeared. Imported the main pool without error.

Hypothesis: Switch port is dead, or flaky, so lost the network, then acted on bad assumptions without sufficient troubleshooting.

Didn't try a new cable or switch or switch port, assumed it was the mobo's interface when neither idb0 or igb1 worked, bought add-in NIC, and used a new cable to a different switch and it worked, so assumption "proven." But some sort of conflict (?) between the add-in PCIe NIC and the LSI SAS controller resulted in the HDDs on the LSI going missing. Put original mobo interface on the different switch, and it worked; removed the add-in NIC, and the LSI HDDs re-appeared.