Grub error: Checksum verification failed after update

[Rickle]

I am running FreeNas 9.3 stable on a 16GB USB drive. Today I thought I would try the new update feature in the FreeNas GUI. I made sure i was still in the "stable" train.

I backed up my config and simply downloaded the latest updates via the GUI and pressed the install button.
(sorry, i did not take note of all the required updates being installed)
15 minutes later, once all updates were downloaded and installed, the system automatically rebooted I get the following error message on a connected monitor.

Grub loading..
Welcome to GRUB!
error: checksum verification failed.
entering rescue mode..

I am convinced i can fix this by re-installing FreeNas on the USB drive and recovering from my backup but would rather try to identify the issue.

any thoughts?

 

[dlavigne]

Since you've already saved your config, try installing the latest STABLE on the stick. If that also fails, the stick is the issue and an install on a fresh stick will fix it.

 

[DrKK]

Dru, he doesn't want to "fix" it, he already knows how to do that. He wants to understand it.

 

[Ericloewe]

Probably just some random corruption.

 

[Rickle]

Still unable to explain why the update broke my system.
I ran a mem test, hard drive test and tested the USB key. No issues were found.
I ended up re-installing and recovering from my backed up config.
I guess I'll stick with the manual updates for future updates :/

Thanks Guys

 

[cyberjock]

No clue what your hardware is, but if you are using non-ECC RAM a RAM test won't find random bit-flips.

What you are seeing is exactly why threads are around that talk about how to stress test and find the weird cases. Often hardware works fine, but those edge cases can be a killer. And you only have to hit that "one edge case that takes the server down" once a month or so and you'll find life very difficult.

What I've seen is that the vast majority of people with USB problems are buying brands that I'd never trust. I avoid all non-name brands and Kingston. I've used (and I am currently using) Corsair Voyager USB sticks without problems.

 

[Rickle]

Yeh, that's what's tricky with ZFS, its great until random flipped bits bite you :)

Puns aside, I have an atom cpu with 4gb of so-dimm non ecc. I run the os on a lexar usb disks (I have a spare, just in case) which I have had good luck with, thus far.

I know the implications of running with non ecc memory (I back up regularly) but dud not expect it to harm the boot sector (grub!!!)

I'm just glad to see that this is not a common issue and that I was able to recover relativly unscaved.

 

[cyberjock]

Actually, if you are running with <8GB of RAM, you can expect any kind of problem, up to and including wiping out your pool. That's why the manual says 8GB minimum.

Not to sound like a jerk, but the second someone says they have <8GB of RAM and have some unknown/unexplained problem I immediately lose interest in troubleshooting because more often than not its the RAM. The fix is simple.. meet the minimum requirements. There's a reason why the manual says 8GB of RAM minimum. We know stuff is horribly broken and AFU when you have less. We also have zero interest in troubleshooting issues that are "self-inflicted" by using inadequate hardware.

 

[liteswap]

Amen to that. I upgraded my installation from 12GB to 16GB RAM and all the little glitches I was seeing disappeared. Add more RAM!

 

[Rickle]

I replaced the USB drive and reloaded. Its been fine ever since.

Thanks for the tip though.

 

[zedfrx]

I had a very similar problem TL;DR as @cyberjock suspected it was due to the Kingston 32gb USB drive

Longer version, I had a generic 16GB USB drive I got from a conference somewhere as my primary, I though for redundancy I'd go for a mirrored boot setup

1. Plug in new kingston USB #1
2. FreeNAS --> System --> Boot Status
3. Attach new kingston USB #1 and wait for resliver
4. Reboot (all ok)
5. Plug in new kingston USB #2
   
6. FreeNAS --> System --> Boot Status
   
7. Replace generic USB with kingston USB #2
8. Wait for resliver
   
9. Reboot..... and....

Grub loading..
Welcome to GRUB!
error: checksum verification failed.​

After much reading and trying different options e.g. grub rescue, reinstall FreeNAS on kingston USB #1, etc. I gave up and rummaged through my desk, found an unused generic 32Gb SB, reinstalled FreeNAS and all was well.