State degraded but Smart is okay, what to do?

[daniels7]

Hello,

today I got such a mail:

Code:

Checking status of zfs pools:
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
MSZone-RAID   21.8T  9.98T  11.8T         -    18%    45%  1.00x  DEGRADED  /mnt
freenas-boot  9.94G  4.10G  5.84G         -      -    41%  1.00x  ONLINE  -

  pool: MSZone-RAID
state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
    Sufficient replicas exist for the pool to continue functioning in a
    degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
    repaired.
  scan: scrub repaired 0 in 17h33m with 0 errors on Sun Aug  2 17:33:53 2015
config:

    NAME                                            STATE     READ WRITE CKSUM
    MSZone-RAID                                     DEGRADED     0     0     0
      raidz1-0                                      ONLINE       0     0     0
        gptid/b7a9dbf3-453f-11e4-89f8-000c2957bde2  ONLINE       0     0     0
        gptid/b816c470-453f-11e4-89f8-000c2957bde2  ONLINE       0     0     0
        gptid/b889316a-453f-11e4-89f8-000c2957bde2  ONLINE       0     0     0
      raidz1-1                                      DEGRADED     0     0     0
        gptid/f434793a-a5be-11e4-96c2-000c2957bdec  ONLINE       0     0     0
        gptid/f49e2b4c-a5be-11e4-96c2-000c2957bdec  ONLINE       0     0     0
        gptid/f4fef090-a5be-11e4-96c2-000c2957bdec  FAULTED      0    95     0  too many errors

errors: No known data errors

Afterwards I did some tests:
(Please see attachment, as the forum didn't let me include it here due to too many characters)

And as far as I can see the disk should be okay, at least if I can trust Smart.

What do you think, should the disk be replaced or not?
And if not, what is causing this behaviour?

Thank you very much for your help :)

Oh and btw. all 6 disks are WD Red 4TB drives.

Greetings,
daniels7

 

[SweetAndLow]

Hardware and smartctl -a /dev/ada$n output

 

[daniels7]

You already have smartctl -a output in former attached file?
And what do you mean by Hardware?
It's nearly identical to your configuration, newest FreeNAS, Intel Xeon E3-1231 V3, 16GB DDR3L ECC RAM, WD-Red 4 TB x6, ASRock E3C226D2I, IBM ServeRaid M1015 in IT Mode.
Everything is Running inside ESXi 6.0 where FreeNAS has direct access to IBM ServeRaid M1015 via VT-d DirectPath (So smart and all stuff is working)

I attached Debug Stuff.

 

[danb35]

Yes, SMART looks OK from what I can see. Make sure the data and power cables are good and connected securely. And fix your SMART test schedule--you're running a short test hourly, which is way too frequently. That isn't necessarily harmful, but it means that the results of any long SMART tests don't show, if they're being run at all. Every 1-3 days is fine for a short test, every 1-4 weeks for a long test. I'd also suggest that your six disks would be better placed in a single RAIDZ2 vdev rather than two RAIDZ1 vdevs, but that's neither here nor there at the moment.

If there's nothing wrong with the cabling, I'd probably try replacing the disk with itself and keeping a close eye on things.

 

[daniels7]

Okay, thank you very much :)
I have two Raid-Z because I started with one and needed to expand the disk space and had no place to put all the data, delete everything and then build a new Raid-Z2. That's why I expanded the existing Raid-Z with another. I don't feel good with it as well, if two disks from one of those vdevs fail I'm done. Do you think it would be a good idea to get an external 8TB drive, put everything there, create a Raid-Z2 and then put everything back? Or will it kill the disks due to the big amount of data read and written two times?

 

[Bidule0hm]

Do you think it would be a good idea to get an external 8TB drive, put everything there, create a Raid-Z2 and then put everything back?

Yes, it's a very good idea ;)

Or will it kill the disks due to the big amount of data read and written two times?

Not a problem, drives are made for this.