Please help which disk is bad

[paulhuynh81]

CRITICAL: The volume VOLUME01 (ZFS) state is DEGRADED: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state.

all disk show it is online

this is the output from my alert panel

• CRITICAL: Device: /dev/da0 [SAT], 32 Currently unreadable (pending) sectors
• CRITICAL: Device: /dev/da0 [SAT], 32 Offline uncorrectable sectors
• CRITICAL: Device: /dev/da0 [SAT], 8 Currently unreadable (pending) sectors
• CRITICAL: Device: /dev/da0 [SAT], 8 Offline uncorrectable sectors
• CRITICAL: Device: /dev/da0 [SAT], ATA error count increased from 2 to 3
• CRITICAL: Device: /dev/da6 [SAT], 144 Currently unreadable (pending) sectors
• CRITICAL: Device: /dev/da6 [SAT], 144 Offline uncorrectable sectors
• CRITICAL: Device: /dev/da6 [SAT], ATA error count increased from 0 to 4
• OK: There is a new update available! Apply it in System -> Update tab.
• CRITICAL: The volume VOLUME01 (ZFS) state is DEGRADED: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state.

 

[gpsguy]

It looks like you are having problems with 2 disks: da0 and da6.

You might want to look at @BiduleOhm 's profile/signature. He has a script that will map device names to serial numbers.

Please post the output of zpool status -v in code tags, to preserve the formatting.


Sent from my phone

 

[Bidule0hm]

Direct link: https://forums.freenas.org/index.ph...d-identification-and-backup-the-config.27365/ ;)

 

[Bidule0hm]

Edit: grrr... this forum engine... let's say it's friday :)

Sorry for the double post.

 

[gpsguy]

Thanks for providing the direct links - I couldn't do it from my phone.


Sent from my phone

 

[Bidule0hm]

No problem ;)

I did it because the @ link to my profile didn't work in your post (because it's 0 (zero), not a O) and I know the forum' search feature isn't exactly the best in the world... :D

BTW @paulhuynh81 can you post the output of the following commands: smartctl -a /dev/ada0 and smartctl -a /dev/ada6 so we can see in what state the drives really are and maybe why they've both failed more or less at the same time.

 

[Bhoot]

snippy snip

Please also provide an output of

Code:

zpool status -v

What Raid setup are you using. Not sure if all drives are showing "online" how is the pool downgraded? I recently had unreadable pending sectors. I received a new disk and manually put the disk offline, only then I got the message

CRITICAL: The volume bhoot (ZFS) state is ONLINE: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state.

Edit: grrr... this forum engine... let's say it's friday :)

Sorry for the double post.

CHEERS

 

[gpsguy]

That's what I asked the OP for yesterday.

That will show us how the volume is configured.

Please also provide an output of

Code:

zpool status -v

 

[Bhoot]

That's what I asked the OP for yesterday.

That will show us how the volume is configured.

sorry.. missed that. :)

 

[paulhuynh81]

here is the output, i have already replace drive ada6 on friday as well

[root@IX-NAS01] ~# zpool status -v
pool: VOLUME01
state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
repaired.
scan: scrub repaired 0 in 15h7m with 0 errors on Sun Dec 20 18:07:55 2015
config:

NAME STATE READ WRITE CKSUM
VOLUME01 DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
gptid/24a118b5-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2537113f-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/25bad772-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2657ab57-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/26dfa415-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2779340c-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/27fb82d8-360a-11e5-9eb4-0015172fbae8 FAULTED 0 51 0 too many errors
gptid/2897c2c0-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/29192059-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/299afe01-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2a20b17a-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2aa527f1-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/016a257d-b643-11e5-9435-0015172fbae8 ONLINE 0 0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Thu Dec 10 06:45:52 2015
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da12p2 ONLINE 0 0 0

errors: No known data errors
[root@IX-NAS01] ~#

 

[paulhuynh81]

OK so i did a zpool clear , but how do i tell the zpool i already insert a new disk and it need start using it again

[root@IX-NAS01] ~# zpool clear VOLUME01
[root@IX-NAS01] ~# zpool status -v
pool: VOLUME01
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://illumos.org/msg/ZFS-8000-2Q
scan: scrub repaired 0 in 15h7m with 0 errors on Sun Dec 20 18:07:55 2015
config:

NAME STATE READ WRITE CKSUM
VOLUME01 DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
gptid/24a118b5-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2537113f-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/25bad772-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2657ab57-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/26dfa415-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2779340c-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
3634112233314258058 UNAVAIL 0 0 0 was /dev/gptid/27fb82d8-360a-11e5-9eb4-0015172fbae8
gptid/2897c2c0-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/29192059-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/299afe01-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2a20b17a-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/2aa527f1-360a-11e5-9eb4-0015172fbae8 ONLINE 0 0 0
gptid/016a257d-b643-11e5-9435-0015172fbae8 ONLINE 0 0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0h0m with 0 errors on Thu Dec 10 06:45:52 2015
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da12p2 ONLINE 0 0 0

errors: No known data errors
[root@IX-NAS01] ~# zpool online VOLUME01
missing device name
usage:
online [-e] <pool> <device> ...
[root@IX-NAS01] ~# zpool online VOLUME01




Thank you everyone for helping me

 

[paulhuynh81]

No problem ;)

I did it because the @ link to my profile didn't work in your post (because it's 0 (zero), not a O) and I know the forum' search feature isn't exactly the best in the world... :D

BTW @paulhuynh81 can you post the output of the following commands: smartctl -a /dev/ada0 and smartctl -a /dev/ada6 so we can see in what state the drives really are and maybe why they've both failed more or less at the same time.

[root@IX-NAS01] ~# smartctl -a /dev/ada0
smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p28 amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/ada0: Unable to detect device type
Please specify device type with the -d option.

Use smartctl -h to get a usage summary

[root@IX-NAS01] ~# smartctl -a /dev/ada6
smartctl 6.3 2014-07-26 r3976 [FreeBSD 9.3-RELEASE-p28 amd64] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/ada6: Unable to detect device type
Please specify device type with the -d option.

Use smartctl -h to get a usage summary

 

[gpsguy]

Use /dev/da6 with the command. Do the same for da0

Did you follow the directions in the manual when you replaced the disk?


Sent from my phone

 

[paulhuynh81]

Use /dev/da6 with the command. Do the same for da0

Did you follow the directions in the manual when you replaced the disk?


Sent from my phone

NO, i did not know there was a step you have to follow to swap out a bad disk, sorry.... most nas i ever work with that was just auto detect and pick up and repair the bad disk once i swap it out

 

[paulhuynh81]

No problem ;)

I did it because the @ link to my profile didn't work in your post (because it's 0 (zero), not a O) and I know the forum' search feature isn't exactly the best in the world... :D

BTW @paulhuynh81 can you post the output of the following commands: smartctl -a /dev/ada0 and smartctl -a /dev/ada6 so we can see in what state the drives really are and maybe why they've both failed more or less at the same time.

here is the attach text file for the smartctl

 

[Bidule0hm]

Ok, these drives have never seen a SMART test, that's bad... that's probably why two drives failed at the same time: I think they didn't failed at the same time but as the first to fail got unnoticed because of lack of tests, you discovered the problem only when the second one failed and seen that the first one was failed too.

NB: sorry for the adaX/daX mixing, I didn't read enough I guess :D

 

[joeschmuck]

NO, i did not know there was a step you have to follow to swap out a bad disk, sorry.... most nas i ever work with that was just auto detect and pick up and repair the bad disk once i swap it out

Oh no, FreeNAS is not even close to that type of NAS device and it requires a good working knowledge of how to operate, setup, and perform routine maintenance tasks. You cannot simply replace the failed drive and watch it repair the pool although it would be a nice touch. I strongly urge you to read the following user guide section for replacing a failed hard drive.
http://doc.freenas.org/9.3/freenas_storage.html#replacing-a-failed-drive

Additionally I noticed that the drive you posted the SMART results for was a SSHD (Hybrid) drive and I'm not certain I would recommend it for a ZFS file system, although I guess it could be fine but I don't know either. Your drive has very few hours on it and if your other drives are the same type, pay attention to see if there is a trend failing.

 

[paulhuynh81]

Can i paid someone to gave me some 1 on 1 remote support? Please PM me..

How can i tell Freenas to replace the bad disk when the disk is already show as online.

 

[Bhoot]

How can i tell Freenas to replace the bad disk when the disk is already show as online.

I don't think youtube video links are allowed here but if you are feeling too lazy to read the manual I feel you shouldn't be using freenas. Just my 2 bit of what I have learnt.
You need to go to storage on top > View Disks. There will be all the disks listed and you need note down the serial number for the disk which is bad.
Next click storage on top again> Click <zpool name> > click volume status (lower side appears as a white page)
Next Select the disk and click "Offline". Once that is done go to the machine and physically replace the disk. Would recommend shutting down the machine as mine did not support hot swap.
Once the machine restarts go back to volume status and click replace. There should be pop up with a drop down already displaying the new hard disk.
For reference check the attached image. The red boxes are what you will need exactly on the storage page.

PS : The screenshot is not of my system. I am not with my freenas machine so couldn't use images of my GUI. Sorry if this belongs to any of the member. Just needed to pick up an image to solve the issue.

 

[Bidule0hm]

I don't think youtube video links are allowed here

They are, but of course don't post crappy videos who suggest to do stupid things ;)

NB: the volume manager button will not be useful here.

 

[Bhoot]

https://www.youtube.com/watch?v=vb5rqQiu0GM
I would just add shutting down the machine while swapping out the hard disk. :)
The video has worked for me so I can vouch for it being a tried and tested way for replacing HDD.