New CRITICAL alert for /usr/local/bin/gdb7111 showing permanent errors

[bandito123]

Got an email reporting there was a critical error this morning. Logged into the Web UI and this is the error text:

Code:

CRITICAL: Nov. 26, 2016, 9:01 a.m. - The boot volume state is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.

The output from pool status -v shows this:

Code:

  pool: freenas-boot
state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
   see: http://illumos.org/msg/ZFS-8000-8A
  scan: scrub repaired 0 in 0h5m with 1 errors on Sat Nov 26 09:24:55 2016
config:

	NAME											STATE	 READ WRITE CKSUM
	freenas-boot									ONLINE	   0	 0	 1
	  mirror-0									  ONLINE	   0	 0	 2
		gptid/902beb21-e879-11e5-9866-0cc47aa99742  ONLINE	   0	 0	 2
		gptid/90568796-e879-11e5-9866-0cc47aa99742  ONLINE	   0	 0	 2

errors: Permanent errors have been detected in the following files:

		//usr/local/bin/gdb7111

So it seems that there is a presumed problem with /usr/local/bin/gdb7111. I've seen a scrub run by cron and I've run one as well manually. Not able to see what is going on with this file. Checked for updates to the OS but on 9.10-U4 at this time. I've not changed any hardware / software after updating to U4 a few days ago and it has been working fine till this morning. Is this the start of a failing OS drive?

 

[DrKK]

This is fairly typical of the USB drive in the early stages of taking a dump. Especially if it's a Kingston ;)

 

[bandito123]

For this I assume I need to just backup the configuration and then re-install from scratch on new boot devices since both show errors?

 

[Robert Trevellyan]

Correct.

 

[bandito123]

For my education, I don't see where you can manage the USB boot devices like you can the storage media in the ZFS pool. Is this type of management in the current U4 version?

 

[darkwarrior]

The boot pool has now ZFS (since 9.3), so it might be a good time to think about mirrored Boot devices ;)
Like this you will not need to go through the hassle of reinstalling FreeNAS when the same things will happen in the future :D

EDit: Here is the link from the official Documentation:
http://doc.freenas.org/9.10/system.html#mirroring-the-boot-device

 

[Stux]

He has mirrored boot devices. Dual URE in the same file meant GDB is toast.

GDB is I think just the debugger.

So time for a boot rebuild.

Try a different brand of USB

 

[darkwarrior]

He has mirrored boot devices. Dual URE in the same file meant GDB is toast.

GDB is I think just the debugger.

So time for a boot rebuild.

Try a different brand of USB

Woops, you are right! My bad. I overlooked that one

 

[Robert Trevellyan]

I don't see where you can manage the USB boot devices like you can the storage media in the ZFS pool

No, they don't show up under Storage, but you can at least set up a scrub interval in the System > Boot tab.

 

[bandito123]

Any time line on the roadmap for the production-ready release of FreeNAS 10? If I'm going to re-install to address this issue I may hold off if the v10 drop is somewhat close on the horizon.

 

[bandito123]

Update: I've solved the problem without re-installing at the time. Steps are:

1. Download the 9.10-U4 iso file
2. Mount it.
3 Find the file base-os-9.10.1-U4-d61691852044f7425790380209f30ef9.tgz and extract it.
4. In the extracted content was the gdb7111 file in question that was reported as corrupt. Copy it over the current file @ /usr/local/bin/
5. Run a scrub of the boot partition from the Web UI.
6. Problem Solved.

I've already bought the 2 new SSD's for the re-install so I will hold out for the FreeNAS 10 release for that. Especially with the new docker concept being available I will need the increase from 16GB USB drives to 256GB SSD's.