Zpool state DEGRADED – Permanent errros have been detected..

[Masterluke]

I have a setup with 4 HDDs, where 1 HDD was damaged. I tried to swap it out and started the resilvering process. After the resilvering was finished it showed this status:

I already tried to delete the two files. In fact they are directories.
I also started one scrub after another in order to try to repair the errors. But the status stayed the same.

Version: FreeNAS-11.2-U7
Model: AMD A4-3400 APU with Radeon(tm) HD Graphics

I hope this are enough infos. Do you have any suggestions on how to repair my NAS?

Nice regards
Luke

 

[toadman]

So normally you'd run a zpool clear (to clear the errors), then a scrub.

Those files are in the .system directory. The rrd- file is probably just monitoring data. The configs- file would have me a bit spooked. If the scrub runs ok you might consider downloading your config and restoring it.

 

[Masterluke]

I ran a zpool clear (+resilvering) and then a scrub. Sadly it changed nothing.
The errors are still there.

How do i download the config and restore it?

 

[styno]

Wait, that does not make sense, a raidz pool is protected from single disk failure and should not have corrupt files in this situation.
Something else is going on here. Are you sure you swapped the correct disk? Can you start a memory check?
Maybe paste the output of 'zpool history'?

 

[Masterluke]

Yes I´m pretty sure it was the right disk, because I ran a few smart tests beforehand to find out which disk was damaged.
I will run new smart tests.

How can I start a memory check?

The zpool history is in the attached file.

 

[styno]

Mhh I was hoping to see something obvious in the history file but nothing really pops up.
Memory checks are typically done with memtest86, started from an usb boot stick.

As the corruption is 'only' in the .system dataset you can try to move that one to your boot pool (webgui - system - system dataset, then pick freenas-boot) You'll lose some history graphs and make sure you have a local copy of the config file when you do so.

Finally, a detach of the offline disk should do the trick to get rid of the 'replacing-2' entry but it still doesn't explain how and why the files became corrupted. Hence my request of doing a memory check.

 

[toadman]

I ran a zpool clear (+resilvering) and then a scrub. Sadly it changed nothing.
The errors are still there.

How do i download the config and restore it?

In the legacy UI it's System > General > Save Config.

Then System > General > Upload Config. (Which will reboot the system.)

 

[Masterluke]

@styno: Here are the results of the MemTest86. It says 0 errors and Tests passed 100%.
Is the note in the screenshot 1 a concern?

@toadman: Is there any chance I could lose data while saving and uploading the config?

 

[Masterluke]

Here is the status of the ada2 and ada3. The ada2 got interrupted in the last 5 long smarttests.
Is that a concerning sign?