NAS won't boot after upgrade

[Juvx]

I just had a Nightmare weekend in terms of hardware.

Started off Saturday with:
Intel Xeon E3-1246 V3
Fractal Define R5
ASUS Maximus VII HERO Motherboard
32 GB (4 x 8 GB) Vengence Pro 2400mhz mem
Enermax 550W 80+bronze moduler PSU
And a bunch of HDDs 2 x 1TB WD Black; 5 x 4TB WE Red; 1 x 2TB WD Green

Had a ton of issues with the MB. The onboard video wasn't working so i had to use a graphics card, the 2 inner (close to CPU) most DIMMs didn't work and just gave me error 55 over and over (memory was good I checked it). I manage to boot it with 16GB on the outer DIMMs and install FreeNAS. Install was successful and even started to play around with it and creating users, etc. However, suddenly the GUI stopped working after a reboot, tried rebooting again and resetting everything. Nothing. I figured freeNAS didn't like my Video Card (geforce gtx670) and I didn't want to have to use it due to power consumption... so I decided to go to Mircocenter and upgrade it and get ECC memory while i'm at it...

So I replaced the MB and Memory
with:
Supermicro X10SLL-F-O 1150 MB and
32GB of Crucial 1600 ECC memory (#CT2KIT102472BD160B)

Put everything back together... and ... Pressed the power button.... no boot... Motherboard LEDs indicate it's getting power but nothing happens when I press the power button.. No fans spinning, no MB beeping, nothing... I tried reseating CPU and Memory, Making sure the pins for the power button were on correctly and even tried to short the pins using a paper clip to see if maybe its the case... Still no Boot... I'm stuck at this point.

Sorry for making this so long but I just wanted to share my nightmare so far... and Now i'm not sure what to do next.

 

[diedrichg]

Try shorting the pins on the power supply 24-pin cable. See if you are getting proper voltage using a multimeter.

Make sure you have the RAM in the proper slots since you are using two sticks. Read the motherboard manual for this info. You may even want to try to boot with just one of the sticks installed to see if it's a bad stick.

 

[diedrichg]

Off topic, but, how do you plan to implement the drives? What ZFS confrontations are you planning?

 

[Juvx]

Try shorting the pins on the power supply 24-pin cable. See if you are getting proper voltage using a multimeter.

Make sure you have the RAM in the proper slots since you are using two sticks. Read the motherboard manual for this info. You may even want to try to boot with just one of the sticks installed to see if it's a bad stick.

I've already tried one stick at a time.. no luck. I used the exact same PSU in the build with the other MB that very morning and it was booting and working just fine its not the PSU. Also, the new supermicro MB has the power LEDs on and steady (which according to the manual it is receiving power) It's just not doing anything with that power lol. No beeps either which really makes me mad. Wish it would beep at me.

I plan on putting the 5x 4TB REDs in raid Z1. the other 3 im not sure what to do with yet. Might do a crazy raid 0 on the 2 x 1 TB blacks. and just leave the 2 TB green as is. Any suggestions? Also i forgot to mention i have a 120 GB SSD for cashing but it isn't hooked up yet.

 

[Juvx]

I've decided to just return the MB and Mem back to microcenter.
Just bought a ASRock C226 WS ATX Server Motherboard LGA 1150 from newegg and a new batch of
Crucial 16GB Kit (8GBx2) DDR3/DDR3L-1600MT/s (PC3-12800) DR x8 ECC UDIMM

Switching the memory Mostly to save money. I noticed they were 90$ cheaper (for the 32GB total) on amazon than microcenter.
I went with that asrock MB because the supermicro is very annoying other than it not booting, it only has a VGA connector and no DVI or HDMI or DP. ..... that's very disappointing for 2015... I know its a server MB but come on...

 

[diedrichg]

No video is on purpose; you don't need it. You can do EVERYTHING to a Supermicro mb through IPMI using a web browser on another computer. With that ASRock board you WILL need a monitor and keyboard hooked up to it to get it set up. An ATX SM board could be had for a few $ more or you could go with the same board you just had. I think you just had bad hardware.

I have a feeling you have not read the hardware suggestions on this forum as well as cyberjock's ZFS primer presentation. You should start there first before you dip your toes in any further and start wasting money.

 

[diedrichg]

RAIDZ1 is extremely frowned upon and you should consider RAIDZ2.

You will want to install/run FreeNAS on your SSD since you have it available.

 

[Juvx]

RAIDZ1 is extremely frowned upon and you should consider RAIDZ2.

You will want to install/run FreeNAS on your SSD since you have it available.

Why so? This will be just for my little home network for some storing of photos, home videos etc (which are also backed up online) and a Plex server. Only having 12 TB out of 5 x 4 :(

 

[Juvx]

No video is on purpose; you don't need it. You can do EVERYTHING to a Supermicro mb through IPMI using a web browser on another computer. With that ASRock board you WILL need a monitor and keyboard hooked up to it to get it set up. An ATX SM board could be had for a few $ more or you could go with the same board you just had. I think you just had bad hardware.

I have a feeling you have not read the hardware suggestions on this forum as well as cyberjock's ZFS primer presentation. You should start there first before you dip your toes in any further and start wasting money.

Please elaborate on why you think that regarding the hardware? What would you do in my situation?

 

[mattbbpl]

Please elaborate on why you think that regarding the hardware? What would you do in my situation?

He's probably just trying to alert you to the fact that hardware concerns in this case are significantly different from typical consumer hardware concerns. Your comment about the video ports was just seen as a red flag to that affect.

FreeBSD doesn't have the driver support Windows does, for example, so you need to make sure that the motherboard and add-ons are FreeBSD compatible. Cases are another good example, as these boards are typically built for rack mount chassis so air flow concerns need to be considered in a tower. It's just.... different.

 

[JDCynical]

No offense intended, but your initial choice of motherboard kind of points to it...

"ASUS Maximus VII HERO Motherboard - ROG high-value gaming ATX-board"

Gaming boards don't have the chipsets to support ECC RAM. Also, these boards tend to have a bunch of things that are not generally needed on a server, like sound and HDMI output, and don't have other things that are useful, like IPMI.

And for drives the size of those reds, putting them into ZRAID1 is definitely not recommended, which you would have read if you had looked at cyberjock's guides, which include links to the 'RAID5 is dead' articles (nutshell: resilvering takes very long time at higher capacities, greatly increasing the risk of another drive failing before redundancy is reestablished).

Servers tend to be a very different beast with very different considerations compared to your typical desktop machine.

 

[Juvx]

No offense intended, but your initial choice of motherboard kind of points to it...

"ASUS Maximus VII HERO Motherboard - ROG high-value gaming ATX-board"

Gaming boards don't have the chipsets to support ECC RAM. Also, these boards tend to have a bunch of things that are not generally needed on a server, like sound and HDMI output, and don't have other things that are useful, like IPMI.

And for drives the size of those reds, putting them into ZRAID1 is definitely not recommended, which you would have read if you had looked at cyberjock's guides, which include links to the 'RAID5 is dead' articles (nutshell: resilvering takes very long time at higher capacities, greatly increasing the risk of another drive failing before redundancy is reestablished).

Servers tend to be a very different beast with very different considerations compared to your typical desktop machine.

It was not out of ignorance that i went with the ASUS.. it was to save money, as i had it laying around already. It was a 300$ MB that was going to waste. I didnt go out and buy it for this build. I knew from the start that the optimal setup would require ECC memory and a server MB that would support it. That's why I chose the CPU, so i could upgrade to one in the future. Which i have already done...

I'm complaining for lack of HDMI and DVI ports because since the board won't boot or post in the first place, IPMI is totally useless.

Either way none of this talk of how terrible my hardware choice was initially has actually helped me in my current issue with the board not posting or booting, yet the LEDs are on and indicate it has power....

 

[Juvx]

He's probably just trying to alert you to the fact that hardware concerns in this case are significantly different from typical consumer hardware concerns. Your comment about the video ports was just seen as a red flag to that affect.

FreeBSD doesn't have the driver support Windows does, for example, so you need to make sure that the motherboard and add-ons are FreeBSD compatible. Cases are another good example, as these boards are typically built for rack mount chassis so air flow concerns need to be considered in a tower. It's just.... different.

I totally understand and agree. Well other than the desktop case point you were trying to make. High end gaming ATX towers have some of the best cooling of any PC/server.. simply due to the extremely high temps from overclocking requirements these cases are designed to handle... both for the CPUs and GPUs.. Frankly its overkill in terms of cooling for any server CPU /MB... There are 2 massive 140mm fans blowing through the HDD bay covering the whole bay. I have it set to a balanced air pressure with 2 fans in and 2 fans to exhaust at the back of the case. This maintains good cooling but prevents too much dust intake. With 8 bays It is a perfect case for a home NAS at a decent price (120$) I have no need for Hot swappable bays or the ability to stack multiple cases which true server cases are designed to do.

 

[mattbbpl]

I totally understand and agree. Well other than the desktop case point you were trying to make. High end gaming ATX towers have some of the best cooling of any PC/server.. simply due to the extremely high temps from overclocking requirements these cases are designed to handle... both for the CPUs and GPUs.. Frankly its overkill in terms of cooling for any server CPU /MB... There are 2 massive 140mm fans blowing through the HDD bay covering the whole bay. I have it set to a balanced air pressure with 2 fans in and 2 fans to exhaust at the back of the case. This maintains good cooling but prevents too much dust intake. With 8 bays It is a perfect case for a home NAS at a decent price (120$) I have no need for Hot swappable bays or the ability to stack multiple cases which true server cases are designed to do.

I'm not trying to argue against a desktop case. A lot of people use them, and they work quite well for them. You covered the case I was attempting to refer to in my last post (pardon the brevity, I posted it between meetings at work) which is that a lot of desktop cases will push a lot of air around, but then FreeNAS users stack 8 drives in those cases and don't actively draw air through that stack.

Take no offense, I wasn't attempting to disparage your efforts (and the others weren't as well, I'm sure). We're hear to help - to lend some of our expertise so you can learn from our experience rather than make the same mistakes yourself.

 

[pirateghost]

I'm complaining for lack of HDMI and DVI ports because since the board won't boot or post in the first place, IPMI is totally useless.

As if the inclusion of those would somehow help a system that doesn't power on?

Also, IPMI is fully functional without the system booting. It just needs to power once to kick the NIC on, and then everything can be controlled through the IPMI. I fail to see how this is a bad thing.

 

[JDCynical]

It was not out of ignorance that i went with the ASUS.. it was to save money, as i had it laying around already. It was a 300$ MB that was going to waste. I didnt go out and buy it for this build. I knew from the start that the optimal setup would require ECC memory and a server MB that would support it. That's why I chose the CPU, so i could upgrade to one in the future. Which i have already done...

I'm complaining for lack of HDMI and DVI ports because since the board won't boot or post in the first place, IPMI is totally useless.

Fair enough on the MB choice, I understand that. My first FreeNAS machine started the same way with an old Asus 'business stable' desktop board, then was moved to the current server board.

 

[TXAG26]

Given you've tried two boards and two sets of ram, I'm thinking your PSU might be bad or your case is grounding out something on the boards when they are installed. Do you have another 400+ watt PSU you could try?

That Supermicro board should have worked. Do you know by any chance whether it had an older v1.x bios version? I think that board needs bios v2.0 or higher for the E3-1246v3 (which has been out at least 12-18 months).

I think it's the PSU though from what you have posted. I've had PSU's that will light up the MB lights (which only require a couple of watts on standby mode), yet not have the amperage to actually power on the board (failed internals in psu). The fact that half of your ram got kicked out with tons of errors also might point to power stability issues.

 

[Juvx]

Just an update, I removed the MB from the case and only connected one stick or RAM and the PSU. Manage to get it to power up the fans and boot... however after hooking up the monitor to the VGA still no post.

The Asrock server MB should arrive today. I have a suspicion that the 32GB of ECC ram i bought is not compatible with the supermicro MB for some reason..

I have another question regarding 120GB SSD i have. Would it benefit my system to implement it in ZIL or L2ARC or is it a total waste? If yes what's my best option for a home network. Ill mostly use this as a file server for lightroom and for storing videos after editing. also for PLEX transcoding.

 

[TXAG26]

No need for ZIL or L2ARC in a home network. Put that SSD in your workstation and use it as a fast drive for working with Lightroom or scratch space.

Install FreeNAS to a good USB thumb drive and boot from that. No need to waste a HDD or SATA connection on a disk as FreeNAS loads to ram once it's booted up.

 

[SweetAndLow]

Just an update, I removed the MB from the case and only connected one stick or RAM and the PSU. Manage to get it to power up the fans and boot... however after hooking up the monitor to the VGA still no post.

The Asrock server MB should arrive today. I have a suspicion that the 32GB of ECC ram i bought is not compatible with the supermicro MB for some reason..

I have another question regarding 120GB SSD i have. Would it benefit my system to implement it in ZIL or L2ARC or is it a total waste? If yes what's my best option for a home network. Ill mostly use this as a file server for lightroom and for storing videos after editing. also for PLEX transcoding.

Usually if you don't know you need l2arc then you don't need it. Wait until you have more ram before looking down that path.