Replacing failed drive in ZFS - Pool is not online

[kaz.williams]

Hello there :)

First off specs, let me know what other info would be useful and I'll include it!
Build: FreeNAS-9.2.1.9-RELEASE-x64
Platform: AMD Sempron(tm) 145 Processor
RAM: 20 GB
4x 4TB drives in a ZFS pool, I think it is a Z1, but I'm not sure how to confirm this, is there a command line way to tell?

One of the drives has apparently gone bad. First I tested the SATA connections and those appear to be fine, as when I swap cables between the bad drive and one of the good drives, the good drive still shows up correctly and the bad drive does not.

Here is the zpool status output:

Code:

[root@freenas ~]# zpool status -v                                               
no pools available  

And when I attempt to zpool import:

Code:

[root@freenas ~]# zpool import                                                 
   pool: NIKUVOLZFS1                                                           
     id: 7711231493565674790                                                   
  state: UNAVAIL                                                               
 status: One or more devices are missing from the system.                       
 action: The pool cannot be imported. Attach the missing                       
        devices and try again.                                                 
   see: http://illumos.org/msg/ZFS-8000-6X                                     
 config:                                                                       
                                                                                
        NIKUVOLZFS1                                   UNAVAIL  missing device   
          gptid/e5d8a4d0-0269-11e5-a419-bc5ff4ea1a9e  ONLINE                   
          gptid/e6749646-0269-11e5-a419-bc5ff4ea1a9e  ONLINE                   
          gptid/e70f2752-0269-11e5-a419-bc5ff4ea1a9e  ONLINE                   
                                                                                
        Additional devices are known to be part of this pool, though their     
        exact configuration cannot be determined. 

So after looking at the user manual and searching these forums, it looks like I need to use the GUI and replace the drive. However in my case, since the drive went bad, it appears that I can't even view the disks at all right now. My volume is still listed in the UI, but when I navigate to "View Disks" nothing shows up:

Is there a command line way to replace a failed drive instead? Or, do I have to do something first to get the pool back "online" for the disks to show back up in the UI so that I can select the bad one and replace it?

Thanks in advance!

 

[Heracles]

Hi,

Unfortunately, I did not worked with FreeNAS 9... I started seriously with 11...

From what you posted, I am afraid that your pool may have been built without redundancy...

Code:

root@Atlas[~]# zpool status
  pool: JBCloud
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:59:08 with 0 errors on Fri Apr 10 22:59:10 2020
config:

    NAME                                            STATE     READ WRITE CKSUM
    JBCloud                                         ONLINE       0     0     0
      mirror-0                                      ONLINE       0     0     0
        gptid/42003674-d1db-11e9-94f5-a0369f858d9c  ONLINE       0     0     0
        gptid/9bca240e-d43f-11e9-9f1e-a0369f858d9c  ONLINE       0     0     0
      mirror-1                                      ONLINE       0     0     0
        gptid/529662d0-d1db-11e9-94f5-a0369f858d9c  ONLINE       0     0     0
        gptid/be00368a-d43f-11e9-9f1e-a0369f858d9c  ONLINE       0     0     0
      mirror-2                                      ONLINE       0     0     0
        gptid/633484c3-d1db-11e9-94f5-a0369f858d9c  ONLINE       0     0     0
        gptid/e7442b03-d43f-11e9-9f1e-a0369f858d9c  ONLINE       0     0     0
      mirror-3                                      ONLINE       0     0     0
        gptid/eedeafa0-d2a4-11e9-9f1e-a0369f858d9c  ONLINE       0     0     0
        gptid/699da698-d440-11e9-9f1e-a0369f858d9c  ONLINE       0     0     0


root@Hades[~]# zpool status
  pool: Cloud-DR
 state: ONLINE
  scan: scrub repaired 0 in 0 days 01:01:05 with 0 errors on Wed Apr  1 23:01:11 2020
config:

    NAME                                            STATE     READ WRITE CKSUM
    Cloud-DR                                        ONLINE       0     0     0
      raidz2-0                                      ONLINE       0     0     0
        gptid/eda81336-0b48-11e9-891d-bc305be5d3f0  ONLINE       0     0     0
        gptid/ef7afec5-0b48-11e9-891d-bc305be5d3f0  ONLINE       0     0     0
        gptid/f170c0ba-0b48-11e9-891d-bc305be5d3f0  ONLINE       0     0     0
        gptid/f3467728-0b48-11e9-891d-bc305be5d3f0  ONLINE       0     0     0
        gptid/f4fb0ac1-0b48-11e9-891d-bc305be5d3f0  ONLINE       0     0     0

vDevs are named after the kind of redundancy they are built with. Yours is not named anything special. Is it meaningful or just a difference between v9 and v11 ? I don't know...

But there is also the fact that with 3 perfectly fine drives out of 4, a Raid-Z1 pool would load in degraded status while yours don't load at all... Exactly like a pool without redundancy...

So if indeed your pool had no redundancy... well... nothing more to say than "Sorry for your loss..."

 

[kaz.williams]

Understood, that is possible, unfortunately I forget how exactly the pool was setup in the first place or what I selected. Other than the clue that it's not showing up in degraded status, is there any command line way to figure out what type of pool it is?

Sounds like if it was a Z1, then the absence of one of the drives should not cause what I'm seeing, right? Like I could just literally unplug one of the drives and it should still show up in a degraded status right? Good to know.

 

[Heracles]

Sounds like if it was a Z1, then the absence of one of the drives should not cause what I'm seeing, right?

Exactly. As of now, from FreeNAS' point of view, one of your drive is already missing. So do not remove another one... But a Raid-Z1 vDev will survive that situation and will work while yours does not. The vDev that stop working as soon as 1 drive is missing is the one without any redundancy and in this case, its content is gone.

 

[kaz.williams]

Thanks for the info. I'm tackling it from a different angle then, the HDD itself is completely silent when plugged in, no spinning or beeps or scratches or anything. So maybe it's a PCB short, gonna attempt to either resolve the short or swap with a donor PCB as a final hail mary. Either way lack of redundancy is my fault! I could have sworn I chose the Z1 option.... but that was years ago so who knows lol. Anyway thanks!

 

[Heracles]

Good luck recovering your data. If indeed you can revive that drive, ensure to backup your pool ASAP on anything you have.