Options for Degraded Zpool Raidz1-0 2 files corrupted

[mysticpete]

Hi Team

Can someone give me a hand with this one, shell is showing that two files in the pool have been corrupted, but not that its a drive related problem, rather the pool itself,.
here's my output

root@freenas:/ # zpool status
pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0 days 00:01:20 with 0 errors on Wed Sep 16 03:46:20 2020
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
ada5p2 ONLINE 0 0 0

errors: No known data errors

pool: media
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub repaired 0 in 0 days 04:13:54 with 2 errors on Sat Sep 19 16:20:59 2020
config:

NAME STATE READ WRITE CKSUM
media DEGRADED 2 0 0
raidz1-0 DEGRADED 2 0 0
5447227299327320603 REMOVED 0 0 0 was /dev/gptid/c8198192-42f6-11e3-b481-c8cbb8c52f20
gptid/c8f8d91d-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/c98173f8-42f6-11e3-b481-c8cbb8c52f20 ONLINE 2 0 0
gptid/ca025a49-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/304c50d2-ff29-11e6-87ad-c8cbb8c52f20 ONLINE 0 0 0

errors: 2 data errors, use '-v' for a list
root@freenas:/ # zpool status -v
pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0 days 00:01:20 with 0 errors on Wed Sep 16 03:46:20 2020
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
ada5p2 ONLINE 0 0 0

errors: No known data errors

pool: media
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub repaired 0 in 0 days 04:13:54 with 2 errors on Sat Sep 19 16:20:59 2020
config:

NAME STATE READ WRITE CKSUM
media DEGRADED 2 0 0
raidz1-0 DEGRADED 2 0 0
5447227299327320603 REMOVED 0 0 0 was /dev/gptid/c8198192-42f6-11e3-b481-c8cbb8c52f20
gptid/c8f8d91d-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/c98173f8-42f6-11e3-b481-c8cbb8c52f20 ONLINE 2 0 0
gptid/ca025a49-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/304c50d2-ff29-11e6-87ad-c8cbb8c52f20 ONLINE 0 0 0

errors: Permanent errors have been detected in the following files:

media/media/movies/movies_kids@auto-20200301.0100-2w:/Ice Age Continental Drift/Ice Age Continental Drift-1.m4v
media/media/software@auto-20200301.0100-2w:/HP_Backup/Users/Peter/Pictures/2014-05-04/MVI_0913.MOV

Can someone advise what are my options for repair?

Thanks in advance
Pete

 

[Alecmascot]

It seems that you have one drive missing from the array and another is giving read errors.
The array is at risk as it is RaidZ-1.
I suggest you replace the missing drive and when the resilver is finished you look at read errors.
Please post console cut/pastes in code tags to make them easier to read.

 

[sretalla]

@auto-20200301.0100-2w

Since both of your errors are in snapshots and those snapshots should be expired by now according to the timestamps (March 1st 2020 with 2 week Expiry), you should just destroy those snapshots (and consider looking into the setup of the snapshot job as the automatic removal isn't working).

 

[mysticpete]

Since both of your errors are in snapshots and those snapshots should be expired by now according to the timestamps (March 1st 2020 with 2 week Expiry), you should just destroy those snapshots (and consider looking into the setup of the snapshot job as the automatic removal isn't working).

Thanks I have deleted the old ones, not sure why they are not auto deleting after the two weeks....

 

[mysticpete]

It seems that you have one drive missing from the array and another is giving read errors.
The array is at risk as it is RaidZ-1.
I suggest you replace the missing drive and when the resilver is finished you look at read errors.
Please post console cut/pastes in code tags to make them easier to read.

This is what I get when listing the zpool now, there are 4 2TB drives and an SSD with the freenas software on it.

root@freenas:/ # zpool status -xv
pool: media
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub in progress since Sat Sep 19 18:47:51 2020
3.30T scanned at 456M/s, 2.56T issued at 354M/s, 4.02T total
0 repaired, 63.73% done, 0 days 01:11:58 to go
config:

NAME STATE READ WRITE CKSUM
media DEGRADED 5 0 0
raidz1-0 DEGRADED 5 0 0
5447227299327320603 REMOVED 0 0 0 was /dev/gptid/c8198192-42f6-11e3-b481-c8cbb8c52f20
gptid/c8f8d91d-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/c98173f8-42f6-11e3-b481-c8cbb8c52f20 ONLINE 5 0 0
gptid/ca025a49-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/304c50d2-ff29-11e6-87ad-c8cbb8c52f20 ONLINE 0 0 0

errors: Permanent errors have been detected in the following files:

<0x1016>:<0x2cb>
<0x1020>:<0x3752>

This looks a bit more serious now, as you can see I'm doing a scrub right now.

 

[mysticpete]

Scrub finished,not good news.....

root@freenas:/ # zpool status -xv
pool: media
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub repaired 0 in 0 days 04:07:49 with 3 errors on Sat Sep 19 22:55:40 2020
config:

NAME STATE READ WRITE CKSUM
media DEGRADED 5 0 0
raidz1-0 DEGRADED 5 0 0
5447227299327320603 REMOVED 0 0 0 was /dev/gptid/c8198192-42f6-11e3-b481-c8cbb8c52f20
gptid/c8f8d91d-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/c98173f8-42f6-11e3-b481-c8cbb8c52f20 ONLINE 5 0 0
gptid/ca025a49-42f6-11e3-b481-c8cbb8c52f20 ONLINE 0 0 0
gptid/304c50d2-ff29-11e6-87ad-c8cbb8c52f20 ONLINE 0 0 0

errors: Permanent errors have been detected in the following files:

<0x1016>:<0x2cb>
<0x1020>:<0x3752>

 

[mysticpete]

All fixed, rolled back with a snapshot cleaned all the old snapshots and restarted all good again.