Hey folks,
I am in a not great situation where I have a raidz1 pool which has a failed disk. Normally this wouldn't be a problem - just replace the disk with a spare, let the pool resilver, and move on with life. However, in this situation that process didn't work out as well as I had hoped.
Basically, I aready had hot spares and as far as I knew the spare had been added to the pool and the pool had resilvered, but somehow I ended up with the "failed" disk still online in the pool. This persisted for quite some time, but then another disk died, and the pool wouldn't import due to "I/O error" and zfs told me to destroy and recreate the pool from a backup. Of course, I do not have a backup (yes, lesson learned, that mistake won't be repeated, please do not waste everyone's time telling me I should have had a backup). I had already bought some new much larger disks, and was in the process of copying data off to the new pool, but I didn't get finished doing that before this disaster struck. :(
Anyway, I would still like my data back, so I took images of all of the drives in the failed pool and found the drive that was toast (as it wouldn't come online at all, so I could not make an image of it), and sent it off to a data recovery service hoping they could image the disk and send it back to me, whereupon I could simply mount the drive images in a VM and mount the pool and recover the data.
I got the image back and it looks correct as far as I can tell, but I still get the same I/O error when trying to import the pool. Is there any way I can recover this data or am I out of luck?
Here are a few more details from trying to zpool import:
I am also aware of a Windows recovery tool called ReclaiMe Pro which I have an evaluation license for and am letting it chew through looking for ZFS data at the moment but it's really, really slow and I don't know that I trust it to resolve the problem either, so any pointers here at all would be very helpful.
Thanks for any help or advice anyone may be able to provide! Please let me know if you have any further questions or if there's any information I can provide which might help.
I am in a not great situation where I have a raidz1 pool which has a failed disk. Normally this wouldn't be a problem - just replace the disk with a spare, let the pool resilver, and move on with life. However, in this situation that process didn't work out as well as I had hoped.
Basically, I aready had hot spares and as far as I knew the spare had been added to the pool and the pool had resilvered, but somehow I ended up with the "failed" disk still online in the pool. This persisted for quite some time, but then another disk died, and the pool wouldn't import due to "I/O error" and zfs told me to destroy and recreate the pool from a backup. Of course, I do not have a backup (yes, lesson learned, that mistake won't be repeated, please do not waste everyone's time telling me I should have had a backup). I had already bought some new much larger disks, and was in the process of copying data off to the new pool, but I didn't get finished doing that before this disaster struck. :(
Anyway, I would still like my data back, so I took images of all of the drives in the failed pool and found the drive that was toast (as it wouldn't come online at all, so I could not make an image of it), and sent it off to a data recovery service hoping they could image the disk and send it back to me, whereupon I could simply mount the drive images in a VM and mount the pool and recover the data.
I got the image back and it looks correct as far as I can tell, but I still get the same I/O error when trying to import the pool. Is there any way I can recover this data or am I out of luck?
Here are a few more details from trying to zpool import:
Code:
thalin@recovery:~$ sudo zpool import pool: stor id: 14530119514345764826 state: ONLINE status: The pool was last accessed by another system. action: The pool can be imported using its name or numeric identifier and the '-f' flag. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-EY config: stor ONLINE raidz1-0 ONLINE 5428dd92-88a5-11e8-9e0e-14feb5c73810 ONLINE 59c71c4b-88a5-11e8-9e0e-14feb5c73810 ONLINE spare-2 ONLINE 70112329-45fb-4468-b96d-eca86e60e09f ONLINE 38ccd909-913f-11e8-8547-14feb5c73810 ONLINE 650a4922-88a5-11e8-9e0e-14feb5c73810 ONLINE 6abbada0-88a5-11e8-9e0e-14feb5c73810 ONLINE 7058c7c5-88a5-11e8-9e0e-14feb5c73810 ONLINE spares 38ccd909-913f-11e8-8547-14feb5c73810 thalin@recovery:~$ sudo zpool import -f stor cannot import 'stor': I/O error Destroy and re-create the pool from a backup source.
I am also aware of a Windows recovery tool called ReclaiMe Pro which I have an evaluation license for and am letting it chew through looking for ZFS data at the moment but it's really, really slow and I don't know that I trust it to resolve the problem either, so any pointers here at all would be very helpful.
Thanks for any help or advice anyone may be able to provide! Please let me know if you have any further questions or if there's any information I can provide which might help.