All data gone. Accidential zfs destroy

[lancee]

Hi all - first post and a suberb subject line - yes...but i kinda have a severe problem.
I wanted to delete some snapshots and accidentially deleted the dataset
zfs destroy -r sdasd/123

Is there any way to recover the deleted dataset and its files?

thanks
lancee

 

[cyberjock]

If you rollback your pool's txg to before the delete, you might be able to get it back. But I can't find anyone that's actually restored a dataset after destroying it.

 

[lancee]

Hi - thanks for the hint. I managed to roll back the txg and can see the dataset again. I also see the files - bu unfortunately i cannot copy them away - I/O error - any ideas?

 

[cyberjock]

Can you provide the commands that you used to roll back the txg? and how you derived those commands?

 

[lancee]

Hey - sure and by the way I have the disks now here at my home. I am trying to dd one of the disks to try my data recovery efforts. Do you know any specialties about DDing a single ZFS disk from a RAID1 in FreeNAS or Solaris...

I discovered two ways:

• I already tried this in a VM. Roll back TXGs with -T operator from this post: (http://lists.freebsd.org/pipermail/freebsd-hackers/2013-July/043113.html)
• Forensic Script (not tested yet) http://www.solarisinternals.com/wiki/index.php/ZFS_forensics_scrollback_script

Here is what I tried so far: the two scenarios:
Scenario A Case 1) Lab setup with VMs and leave mirror running without splitting disks up
What I did so far: (reproduction of the real case in the prod enviroment - its a NAS Server for pictures in a photo store)
Installed FreeNAS 8.3.0 in a VM
Two VDisks (with 10GB)
Raid1 - ZFS
I created a pool "datastore2"
I created a dataset "foto2"
I copy a file blabla.mp3
I made "zfs destroy datastore2/foto2" like on the real box
Reboot FreeNAS
open shell: "zpool history -il" then "zdb" to check for add txgs. And took a TXG from the history before the "destroy" command "86"
shell:"zpool export datastore2"
shell" zpool import -N -o readonly=on -f -R /mnt -F -T 86 datastore2"
I managed to import the pool with its prior state.
"zfs list" shows the pool and dataset "foto2"
I saw the files and tried to copy the file - but got IO errors
Scenario A Case 2) What I am trying to do with the real data
I did every thing as mentioned above - but I only took one of the disks because I wanted to save the second one.
I came to the part when I wanted to use a prior "destroy" TXG - but I always ran into the IO error when importing the zpool.
I managed to rollbacl older TXGs but only - after - the destroy command
At the moment I am not sure what to do next.
I am just preparing a box with opensolaris to try the second method with the forensics script.
the below mentioned commands did not work in FreeNAS:
I had good luck with ZFS recovery with the following approach:

1) Use zdb to identify a TXG for which the data structures are intact

2) Select recovery mode by loading the ZFS KLD with "vfs.zfs.recover=1"
set in /boot/loader.conf

3) Import the pool with the above -T option referring to a suitable TXG
found with the help zdb.

The zdb commands to use are:

# zdb -AAA -L -t <TXG> -bcdmu <POOL>

 

[cyberjock]

Actually, if you try to use ZFS and are using the newest feature flags that FreeNAS has, solaris has no support as far as I know. So you won't even be able to try. You'll know if it doesn't work, it'll flat out tell you that you can't do it.

 

[cyberjock]

Are you running FreeNAS 8.3.0?

 

[lancee]

so would you try it with a newer FreeNAS version or with 8.3.0 running on the original box?
Has FreeNAS the DD command to clone one of the disks?

PS: Yes the prod FreeNAS is 8.3.0 x64

 

[lancee]

yes - running 8.3.0 64bit.
Any suggestions how to move on with the dd of the orig disk? Install OpenSolaris or use FreeNAS to clone?

 

[cyberjock]

You mean if you want to make a copy of your untouched disk? I'd do a dd if=/dev/olddisk of=/dev/newdisk bs=1m and let it run until it finishes. Should be able to do that from FreeNAS.

If you were using 8.3.0 before you should stick with 8.3.0 for this task. Normally I'd tell people to go to 9.x. But in your case, definitely stick with 8.3.0. You *could* try 8.3.2, but don't go to 9.x at all.

 

[lancee]

Ok. So I'll go with 8.3 for the dd Command.
Is there anything Special to consider. Use safe Mode or surpress auto importing of pools if needed?

Ps: how can i get the dev namens? /dev/ad0 ad1 etc for the dd.

 

[cyberjock]

The pool shouldn't be imported. For the purpose of duplicating disks I'd just make a fresh FreeNAS disk so that it doesn't import your pool and you don't have to mess up your config.

 

[lancee]

the dd went through - I now have 2GB of free space then 1.8TB of prim partition and the rest is free space. Looks good - and now? :D

 

[cyberjock]

Not sure. You did what I was going to try I think. Damn I wish you had done snapshots.

Sent from my DROID BIONIC using Tapatalk

 

[lancee]

I have the disk now unmounted in the box. I am not sure if I should try to import the pool first in the "destroyed" state or leave is as is and try to get the uberblocks with the fornesic script. Referring to that script and how to - I need now the block size and total bytes of the file_system. Any ideas?
http://www.solarisinternals.com/wiki/index.php/ZFS_forensics_scrollback_script


PS: I had snapshots that was the problem :D Wanted to del these instead it destroyed the whole dataset...

 

[cyberjock]

Block size should be 128kbytes if you made the pool from the GUI. Size should be the disk minus 2GB plus any sectors that werent on a boundary.

Sent from my DROID BIONIC using Tapatalk

 

[lancee]

OK thanks - I made it from the GUI.

Would look like this: ./zfs_revert-0.1.py -bs=128 -tb=yxyxyxyy test_filesystem
But I am unsure what the author means by "test_filesystem" I only have the device so far and no FS.

 

[cyberjock]

Me neither. Never used the script you mentioned. Never really heard of it I don't think.

Sent from my DROID BIONIC using Tapatalk

 

[lancee]

Tried the script did not find anything until now.
Tried the rollback of TXGs - says one device unavailable - cloning both Raid1 disks now.

 

[lancee]

I am also wondering why FreeNAS leaves 2GB of free space on the disks? Is this normal? The 2GB are unsused and then the data partition begins.