SOLVED Power Company Scorched My Boot DOM

[Nipomo]

I'm posting this in case it can help others with the same problem in the future.

Power company recently delivered a momentary over-voltage condition with lots of noise to my house. I have a good UPS, but some of the noise got through and broke my box. Every time I booted, I would get "getty repeating too quickly on port..." with resets every 30 seconds. The console menu was unavailable.

Using the bootloader, I was able to drop back to a previous version of FreeNAS and get the box booted. I scrubbed the boot drive but the box still wouldn't boot up on the latest FreeNAS version. Since the error seemed to be OS version based, I assumed the problem was probably a corrupted boot DOM.

Ordered a new 32G boot DOM from ADATA, downloaded a new copy of the config, removed the old DOM and installed the new one. I had a USB powered DVD drive (Yay!) and I burned the latest ISO. Booted the machine from DVD player and successfully installed version 9.10.2. Machine booted to the new OS and I uploaded my latest config. After another reboot, my NAS was back in action.

If this ever happens in the future, I'll be sure to have a good thumbdrive available as a temporary boot drive while I order a new DOM. Also, my IXSystems FreeNAS Mini has only one Molex plug and it's being used. The power wires for the supplied DOM were built into the one existing Molex plug and wouldn't fit my new DOM. So, if you replace your DOM, you may need to buy a Molex splitter.

Jim
IXSystems FreeNAS Mini
4 drives, 24TB total, 11 TB useable
32G ram
Intel Atom CPU C2750 @ 2.40GHz

 

[JWTech]

I'm posting this in case it can help others with the same problem in the future.

Power company recently delivered a momentary over-voltage condition with lots of noise to my house. I have a good UPS, but some of the noise got through and broke my box. Every time I booted, I would get "getty repeating too quickly on port..." with resets every 30 seconds. The console menu was unavailable.

Using the bootloader, I was able to drop back to a previous version of FreeNAS and get the box booted. I scrubbed the boot drive but the box still wouldn't boot up on the latest FreeNAS version. Since the error seemed to be OS version based, I assumed the problem was probably a corrupted boot DOM.

Ordered a new 32G boot DOM from ADATA, downloaded a new copy of the config, removed the old DOM and installed the new one. I had a USB powered DVD drive (Yay!) and I burned the latest ISO. Booted the machine from DVD player and successfully installed version 9.10.2. Machine booted to the new OS and I uploaded my latest config. After another reboot, my NAS was back in action.

If this ever happens in the future, I'll be sure to have a good thumbdrive available as a temporary boot drive while I order a new DOM. Also, my IXSystems FreeNAS Mini has only one Molex plug and it's being used. The power wires for the supplied DOM were built into the one existing Molex plug and wouldn't fit my new DOM. So, if you replace your DOM, you may need to buy a Molex splitter.

Jim
IXSystems FreeNAS Mini
4 drives, 24TB total, 11 TB useable
32G ram
Intel Atom CPU C2750 @ 2.40GHz

What type of UPS were you using?

Also, was it stepped, or was it pure sine wave?

 

[DrKK]

for 99.911823% of people on this forum, a stepped UPS is more than sufficient, and can be run indefinitely without risk to the box.

 

[JWTech]

for 99.911823% of people on this forum, a stepped UPS is more than sufficient, and can be run indefinitely without risk to the box.

Thanks for the input, but I don't believe I had asked that.

In this instance whatever was in use obviously was not sufficient.

 

[DrKK]

Also, was it stepped, or was it pure sine wave?

You asked this. Implying that the answer is somehow relevant to this user's concerns. I don't believe it is, and I don't want your asking him to make him think he has to somehow spend 3x as much money on a UPS than the rest of us.

It might be nice to know the make and model of his UPS, however, I will agree with that. @Nipomo could you tell us please?

In any case, the problem the user is describing may not, coincidentally, be directly related to the power issue, despite the contemporaneous nature of the events. We have had a few users describe the issue "getty repeating too frequently" or whatever it is, and those users did not have any power issue or loss of power.

 

[m0nkey_]

Power company recently delivered a momentary over-voltage condition with lots of noise to my house. I have a good UPS, but some of the noise got through and broke my box.

Sounds like your house wiring may not be up to scratch. Even the most basic UPS should have been able to avoid this from happening. The only time I've seen it previously is when the mains earth isn't actually going to earth.

If your UPS has any kind of earth fault indicator, it'll be a good idea to see if it's on. I know for a fact my Cyberpower UPS has a earth fault protection indicator, and it used to be on in my old apartment. When I moved, I had the earth tested and have no such problem.

Most UPS's come with a warning to that effect, if the earth has a fault, it may not be able to protect against surges.

 

[DrKK]

mains earth

This is called "ground" in the first world, sir.

 

[m0nkey_]

This is called "ground" in the first world, sir.

Sorry, I only know British electrical terms. Some 'merican terms are still new to me.

 

[DrKK]

Some 'merican terms are still new to me.

well, we invented the idea of, and were the first to implement, the electrification of homes. The first home that was electrified was that of J.P. Morgan, personally electrified by Edison himself.

So, I believe that gives us priority for home electrification terms sir.

So, it's called "[earth] ground". QED. Your views on the matter are hereby pre-empted sir!!!!

 

[m0nkey_]

well, we invented the idea of, and were the first to implement, the electrification of homes. The first home that was electrified was that of J.P. Morgan, personally electrified by Edison himself.

So, I believe that gives us priority for home electrification terms sir.

So, it's called "[earth] ground". QED. Your views on the matter are hereby pre-empted sir!!!!

nerd.

 

[DrKK]

nerd.

We mock what we don't understand.

 

[m0nkey_]

Let's just bring this thread back down to earth (pun intended).

Look for any electrical faults. You can usually buy a simple tester from any hardware store to test for wiring faults.

 

[DrKK]

I did notice you can now set ups.conf *and* upsd.conf, in the services menu. That was a long-needed change.

 

[Nipomo]

What type of UPS were you using?

Also, was it stepped, or was it pure sine wave?

The UPS is an APC XS-1500, stepped. A better USP might have prevented my problem. On the other hand, I may have had a weak DOM.

 

[JWTech]

You asked this. Implying that the answer is somehow relevant to this user's concerns. I don't believe it is, and I don't want your asking him to make him think he has to somehow spend 3x as much money on a UPS than the rest of us.

I asked him, not you. You randomly interrupted with a blatantly non-factual opinion that was clearly condescending, and since then have notably derailed this thread.

And yes, the UPS is clearly relevant, and pure sine wave systems only cost marginally more with notable improvements to preventing equipment failure with sensitive components (part of the reason they exist).

I don't want your asking him

^^ That is actually quite concerning. If you were a moderator, would you have deleted my question?

 

[Nipomo]

Sounds like your house wiring may not be up to scratch. Even the most basic UPS should have been able to avoid this from happening. The only time I've seen it previously is when the mains earth isn't actually going to earth.

If your UPS has any kind of earth fault indicator, it'll be a good idea to see if it's on. I know for a fact my Cyberpower UPS has a earth fault protection indicator, and it used to be on in my old apartment. When I moved, I had the earth tested and have no such problem.

Most UPS's come with a warning to that effect, if the earth has a fault, it may not be able to protect against surges.

Thanks for the advice. My grounds (earth) are fine, all outlets are 3 conductor w/ neutral tied to ground at the power box. Everything worked well, the UPS sensed the over-voltage and told the NAS to shut down. The NAS shut down fine but failed on re-boot. I'm pretty sure some stray RF got into my box and corrupted the DOM.

 

[m0nkey_]

I asked him, not you. You randomly interrupted with a blatantly non-factual opinion that was clearly condescending, and since then have notably derailed this thread.

To be fair, this is a forum where everyone is welcome to comment. Whether the answer is right or wrong, it doesn't give you a reason to blast another community member, especially one with the post count @DrKK has. You can rest assured that you will not make friends if you post with that attitude.

And yes, the UPS is clearly relevant, and pure sine wave systems only cost marginally more with notable improvements to preventing equipment failure with sensitive components (part of the reason they exist).

For the majority FreeNAS users, who have a box in their bedroom, basement or whatever, a stepped UPS is sufficient. While having pure sine wave is nice, most power supplies will run just fine on stepped. Some Active PFC supplies will fail to operate, such as the PSUs found in some enterprise class servers. Your average Corsair, SeaSonic or Cooler Master PSUs really won't care.

Any UPS worth it's money will stand up to brown outs and surges. Since we don't know a lot about the condition of the UPS in question, it's very possible the UPS did fail to protect and caused a component to fail.

@Nipomo, it might be worth reaching out to APC to see if you can claim any sort of damage as most UPS's come with some kind of guarantee.

 

[DrKK]

@Nipomo, it might be worth reaching out to APC to see if you can claim any sort of damage as most UPS's come with some kind of guarantee.

Almost positive APC has this. Whether or not the $40 DOM (or whatever) is worth the $80 of hassle you're likely to encounter getting the reimbursement, however, is anyone's guess.

 

[JWTech]

To be fair, this is a forum where everyone is welcome to comment. Whether the answer is right or wrong, it doesn't give you a reason to blast another community member, especially one with the post count @DrKK has. You can rest assured that you will not make friends if you post with that attitude.

I had thought my reaction was reasonable considering. Notably upset but not overly hostile.

Nonetheless, I apologize if I came off a bit harsh.

 

[Nipomo]

To be fair, this is a forum where everyone is welcome to comment. Whether the answer is right or wrong, it doesn't give you a reason to blast another community member, especially one with the post count @DrKK has. You can rest assured that you will not make friends if you post with that attitude.

For the majority FreeNAS users, who have a box in their bedroom, basement or whatever, a stepped UPS is sufficient. While having pure sine wave is nice, most power supplies will run just fine on stepped. Some Active PFC supplies will fail to operate, such as the PSUs found in some enterprise class servers. Your average Corsair, SeaSonic or Cooler Master PSUs really won't care.

Any UPS worth it's money will stand up to brown outs and surges. Since we don't know a lot about the condition of the UPS in question, it's very possible the UPS did fail to protect and caused a component to fail.

@Nipomo, it might be worth reaching out to APC to see if you can claim any sort of damage as most UPS's come with some kind of guarantee.

Thanks, but I've had that UPS for a good many years and I'm well into the 2nd set of backup batteries. This was the first time I've seen something like this and I can't rule out a weak DOM.

It was a little unnerving at first, but FreeNAS is a great OS and since I had a good set of config backups it was pretty easy to recover from. I'm only out the cost of a new DOM and a Molex splitter and I got to use my troubleshooting skillz. DOMs are way better than thumb drives, but they fail just like any SSD. My NAS is happy, I'm happy.