Lost my ZFS pool and I can't seem to get it back.

[Farokh]

I've been using freeness for a while, but never had any problems that I couldn't handle until last night.

I was running 8.0.4 on an old S5000PAL based system and I rebooted it cleanly (or so I thought). When it restarted, all three volumes had invalid GPT partition errors (I think, I can't recall anymore).

I tried to move up to 8.3, since I saw some posts about 8.2 and newer being able to fix some errors that 8.0 couldn't. For some reason, I just couldn't get 8.3 to work on this machine. It would start up and then give me some NMI errors and the drop me to rdb.

So I happened to have some RAM (this machine had only 4 GB), and I was able to bump it up to 16 GB and I installed a fresh 9.3 on a new 16 GB stick and have a running machine. However, my zfs pool is gone, and I can't seem to figure out how to get it back.

From what I was able to find searching around, the gpart list command shows all 3 parts of the zfs pool (I've got 3 disks, each one has 2 parts, a swap of 2GB and a ifs of 1.8TB). The zfs parts show up as da0p2, da1p2 and da2p2.

When I do global status, it only shows 1 of those 3:

Code:

[root@freenas] ~# glabel status
                                      Name  Status  Components

gptid/265acc23-cdbb-11e1-ac72-000e0ce9f064     N/A  da0p1
gptid/26694d4e-cdbb-11e1-ac72-000e0ce9f064     N/A  da0p2
                            ufs/FreeNASs3     N/A  da3s3
                            ufs/FreeNASs4     N/A  da3s4
                            ufs/FreeNASs1a     N/A  da3s1a

There's no mention of da1, or da2.

zpool status just tells me that there are no pools available.
pool import (with or without -F -f -n) spins for about 2 seconds and then exits to the prompt without any output.

If it helps, I still have the 4GB stick with the old 8.0.4 on it.

I've been up all night (it's 8 am here in NY) and I really need to get this fixed asap. I've got about 10 different VMs down because they are on this shared device.

Any pointers would be appreciated.

Thanks!

 

[danb35]

It doesn't look to me like da3 is a ZFS disk at all; it looks instead like it was UFS. What does camcontrol devlist show?

ETA: If this is UFS, 9.3 won't be able to deal with it, as it's gone ZFS-only. You should still be able to use your disks under 9.2 and prior, though.

 

[Farokh]

da3 is UFS, that's the USB stick that's running freenas. It's da0, da1 and da2 that are the ZFS disks. da0 is the only one that shows up in glabel status.

Thanks.

 

[danb35]

Sorry, missed that in your OP and hoped it would be something simple. I'm afraid this doesn't look good.

 

[Farokh]

I'm still looking around to see what I can try, and came across someone mentioning gpart, so I ran that, and here's what I got:

Code:

[root@freenas] ~# gpart show
=>        34  3907029101  da0  GPT  (1.8T)
          34          94       - free -  (47k)
        128     4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)

=>        34  3907029101  da1  GPT  (1.8T)
          34          94       - free -  (47k)
        128     4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)

=>        34  3907029101  da2  GPT  (1.8T)
          34          94       - free -  (47k)
        128     4194304    1  freebsd-swap  (2.0G)
    4194432  3902834703    2  freebsd-zfs  (1.8T)

=>      63  31266753  da3  MBR  (14G)
        63   1930257    1  freebsd  [active]  (942M)
  1930320        63       - free -  (31k)
  1930383   1930257    2  freebsd  (942M)
  3860640      3024    3  freebsd  (1.5M)
  3863664     41328    4  freebsd  (20M)
  3904992  27361824       - free -  (13G)

=>      0  1930257  da3s1  BSD  (942M)
        0       16         - free -  (8.0k)
      16  1930241      1  !0  (942M)

and it shows all the ZFS partitions.

I also realized that I don't remember what I called my pool. Is there a way to find out what the name of the pool was?

Thanks.

 

[Jailer]

This is going to sound like a silly question but did you try the auto import function from the GUI?

 

[Farokh]

Yes, it didn't do anything. It prompts me if it's encrypted, which it isn't, and then after that it simply gives me a blank box for the volume name and all I can do is cancel.

I'm not sure if the the plain Import Volume will let me do something, but I'm afraid to try anything that might cause me to lose data (I'm assuming that I haven't actually lost anything yet).

 

[Apollo]

best bait is to revert to freenas 8 and go from there

 

[cyberjock]

Try this....

# zpool import

Post the output in pastebin and not the forums. ;)

 

[Farokh]

Apollo: I tried running FreeNAS 8, it didn't see the pool either.

cyberjock: zpool import simply exits after about 2 seconds with no message at all.

 

[cyberjock]

Can you post system specs and a debug file?

And are you virtualizing FreeNAS?

 

[Farokh]

cyberjock: If you can tell me exactly what to do to get a debug file, I'll be more than happy to do it (just want to be sure I do the right thing) :).

As for the specs, it's a pretty generic 1U server box with an Intel S5000PAL motherboard with 16GB RAM and 2 Intel Xeon running at 3GHZ. I don't have the machine on at the moment (it makes a ton of noise because of the fans), so I don't recall the exact CPU specs.

Finally, no, there's no virtualizing on this machine. FreeNAS is bare metal.

Thanks.

 

[cyberjock]

The debug file can be generated under the system tab. I think it's System -> advanced -> Save Debug or something like that. My network is all torn down right now so I have no access to any FreeNAS machines to check.

 

[tomekadascientiam]

Honey,

You can pull up the debug by going to system 0-> settings 0-> advanced 0-> save debug

In the event you have issues with getting the debug, and so that we can save from having to probe your debug can give the output for these commands as well:

dmidecode -t 1 | grep -e 'Manufacturer' -e 'Product Name' -e 'Serial Number' | cut -f 2

dmidecode -t 0 | grep -e 'Vendor' -e 'Version' -e 'Release Date' | cut -f 2g

sas2flash -c 0 -listall

sysctl -a | grep mps | grep version

zdb -C | less

 

[Farokh]

OK, I tried that, but it failed from the GUI (the error I got was that there was no such file /var/tmp/ixdiagnose/ixdiagnose.tgz).

I ssh'd in and found stuff in /var/tmp/ixdiagnose/ixdiagnose, so I created a .tgz file from that. I'm uploading it here.

Thanks.