Help with recovery...

[buckroa]

Hi All,

I have Freenas 8.3-release-p10 running on a HP Microserver N36L, with 5 x 2TB Hitachi 5K3000 drives in Raid-Z

One of the drives failed, which I've just received the replacement for....

While waiting on the replacement, a 2nd drive started to fail; I started to see messages such as..

Device: /dev/ada1, Failed SMART usage Attribute: 5 Reallocated_Sector_Ct.
Device: /dev/ada1, FAILED SMART self-check. BACK UP DATA NOW!
Device: /dev/ada1, failed to read SMART Attribute Data​

I shut the server down, and crossed my fingers while waiting for the new drive to arrive.

I have now replaced the first failed drive, with a new drive, hoping the 2nd failing drive will stay alive long enough for me to rebuild the array.

When I power up freenas, when trying to mount local file systems, I get the following messages :

ahcich2: Timeout on slot XX port 0
achich2: is 00000000 cs 00000000 ss 00000000 rs 00000000 tfd 50 00000000 cmd 00000000​

repeated with slightly different numbers each time...

Is there anything I can do to get my data back? or is the 2nd failing drive too far gone.

 

[cyberjock]

Its probably gone forever.

If you know which 2 bad disks you have, you *might* be able to use ddrescue to copy those bad disks to good disks. Do be warned that ddrescue can take days on a single drive.

Now you see why we tell people not to do RAIDZ1.. :/

 

[buckroa]

I know which disk is bad - and I have a blank good one to copy it to...

what's the best way to attempt recovery with ddrescue? Specifically for zfs, if I do manage to copy the data over how do I join that disk to the pool?

 

[cyberjock]

You use ddrescue to copy from the bad disk to the good disk. It'll probably throw LOTS of errors because the disk is failing. Let it finish and be patient. Once done put the "new" disk in your server and boot it up. With a little luck it will add to the pool magically if ddrescue got enough ZFS structures for the zpool to recognize it.

Likewise if it doesn't magically go in the pool then not enough ZFS structures were recovered and your pool is gone without spending some cash on recovery (think car loan kinds of cash).

 

[buckroa]

It looks like recoverdisk is on the freenas usb drive, whereas ddrescue is not - is that a suitable replacement?

 

[cyberjock]

No. ddrescue is a special program, it will probably require you to install linux or FreeBSD on another machine to do this.

 

[Robert Trevellyan]

ddrescue is present on Ubuntu Rescue Remix, which will save the trouble of installing an OS on another machine.