Degraded Volume!

[BORSTE]

Hi All,
Hope someone can assist me here. I have searched previous threads on the forum and have found users experiencing similar problems but not identical.

I get the following system alert:

• CRITICAL: July 24, 2017, 8:14 a.m. - The volume myVOL state is DEGRADED: One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state.

When I view the volumes: (There should be a total of 4 drives 3TB each, setup as ZFS)
ada0 - online
ada1 - online
ada2 - online
10824739011237558411 - unavailable (This status does not change if the old or new disk is inserted)

When I click on the unavailable drive, it only gives me a replace option. When I select the replace option, it asks for a new member disk but there are no options in the dropdown.

Ive tried booting with only 3 drives and changing the bays to ensure that the drive is the problem and it boots fine. I cannot take any of the 3 active drives offline because I get the error:

MiddlewareError: b'Disk offline failed: "cannot offline gptid/23b76ea6-d025-11e5-a77b-3ca82aa0a80c: no valid replicas, "']

My hardware is:
HP Proliant Microserver Gen8
8 GB Ram
4 x 3 TB HDD
FreeNas 11.0-U2

Im new to FreeNas so a basic explanation on how to solve this would really be appreciated.
If you need any further info, please let me know.

Thanks in advance!

 

[danb35]

It appears that one of your four disks has failed.

I cannot take any of the 3 active drives offline because I get the error:

No, of course you can't. You have only one disk's worth of redundancy; if you take a second disk offline, your entire pool will go offline. The likely solution will be to replace the apparently-failed disk as instructed in the manual.

The remaining question is whether the "bad" disk (ada3, presumably) has totally failed, or is only mostly dead. To begin, post the output of camcontrol devlist and let's see if there's even a device listed for the fourth disk.

 

[BORSTE]

It appears that one of your four disks has failed.

No, of course you can't. You have only one disk's worth of redundancy; if you take a second disk offline, your entire pool will go offline. The likely solution will be to replace the apparently-failed disk as instructed in the manual.

The remaining question is whether the "bad" disk (ada3, presumably) has totally failed, or is only mostly dead. To begin, post the output of camcontrol devlist and let's see if there's even a device listed for the fourth disk.

Hi danb35. Apologies for the late response.
Thanks for your input.

Ive tried everything the manual says. I don't get the options to take the disk offline or replace it, furthermore it doesn't pickup the new drive that I insert.

Here is the output:
[root@freenas ~]# camcontrol devlist
<ST3000DM001-1ER166 CC26> at scbus0 target 0 lun 0 (pass0,ada0)
<ST3000DM001-1ER166 CC26> at scbus3 target 0 lun 0 (pass1,ada1)
<SanDisk Cruzer Fit 1.00> at scbus7 target 0 lun 0 (pass2,da0)
[root@freenas ~]#

I now have a further problem. I had a power failure and another drive has bombed out.
Im getting this message and obviously can't access anything:

• CRITICAL: Aug. 4, 2017, 8:51 p.m. - The volume myVOL state is UNKNOWN:

Is there anything I can do or is my data basically lost? I have installed a UPS.

Thanks again for the help!

 

[Stux]

I now have a further problem. ... Is there anything I can do or is my data basically lost?

Until you can get one of those two failed drives to come back, you have no data.

If you connect each disk, one at time, do you see a device each time? Ie shutdown, reboot, camcontrol devlist etc

 

[BORSTE]

Ok, Ive inserted only 1 of the failed drives into the bay (Ive removed all other drives)
Output:
[root@freenas ~]# camcontrol devlist
<SanDisk Cruzer Fit 1.00> at scbus7 target 0 lun 0 (pass0,da0)
[root@freenas ~]#

Output once Ive replaced first failed drive with the 2nd failed drive:
[root@freenas ~]# camcontrol devlist
<SanDisk Cruzer Fit 1.00> at scbus7 target 0 lun 0 (pass0,da0)
[root@freenas ~]#