No pool after boot drive replacement

[d0rk]

I've got a 6x3TB RaidZ2 system. I've tried to replace my USB boot drives with a small SSD boot drive. I was running 11.1-RC1 on the USB drives and created the SSD drive with the ISO I downloaded for the same version. I had encryption turned on for the drives in the single pool I had created.

After booting from the SSD, I restored my old backed up configuration via the web UI. After restoring (and rebooting), the volume showed as locked and I attempted to unlock it via the web UI. That didn't work (I didn't save the error, but it was a generic failure message). I read that you can usually detach and then import the volume when this happens. Detach worked but importing failed. The web UI now doesn't show any volumes available.

So, I flipped back to boot from the USB drives. The volume shows as locked -- just as it did when I first booted from the SSD. I've flipped back and forth a few times trying different commands but I haven't been able to import the pool from either boot media.

Here's what the system looks like right now, booted from the SSD:

zpool import
(no output)

camcontrol devlist
<ATA ST3000DM001-9YN1 CC4B> at scbus0 target 0 lun 0 (pass0,da0)
<ATA ST3000DM001-1ER1 CC25> at scbus0 target 1 lun 0 (pass1,da1)
<ATA WDC WD30EFRX-68A 0A80> at scbus0 target 2 lun 0 (pass2,da2)
<ATA WDC WD30EFRX-68A 0A80> at scbus0 target 3 lun 0 (pass3,da3)
<ATA WDC WD30EFRX-68A 0A80> at scbus0 target 4 lun 0 (pass4,da4)
<ATA WDC WD30EFRX-68E 0A80> at scbus0 target 5 lun 0 (pass5,da5)
<ATA SPCC Solid State 21.1> at scbus0 target 7 lun 0 (pass6,da6)

gpart show
=> 40 5860533088 da0 GPT (2.7T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338688 2 freebsd-zfs (2.7T)
5860533120 8 - free - (4.0K)

=> 40 5860533088 da1 GPT (2.7T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338688 2 freebsd-zfs (2.7T)
5860533120 8 - free - (4.0K)

=> 40 5860533088 da2 GPT (2.7T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338688 2 freebsd-zfs (2.7T)
5860533120 8 - free - (4.0K)

=> 40 5860533088 da3 GPT (2.7T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338688 2 freebsd-zfs (2.7T)
5860533120 8 - free - (4.0K)

=> 40 5860533088 da4 GPT (2.7T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338688 2 freebsd-zfs (2.7T)
5860533120 8 - free - (4.0K)

=> 40 5860533088 da5 GPT (2.7T)
40 88 - free - (44K)
128 4194304 1 freebsd-swap (2.0G)
4194432 5856338688 2 freebsd-zfs (2.7T)
5860533120 8 - free - (4.0K)

=> 40 117231328 da6 GPT (56G)
40 1024 1 bios-boot (512K)
1064 117230296 2 freebsd-zfs (56G)
117231360 8 - free - (4.0K)

geli attach -p -k geli.key da0p2
geli: Cannot read metadata from da0p2: Invalid argument.

I'd appreciate any advice on what I can do to recover the pool.

Thanks!
Chris

 

[joeschmuck]

You said "USB Drives" so I'm assuming you mean you have a mirrored setup. Did you try to boot from the second USB drive?

I've got to tell you, this is a perfect example of why I don't like encrypted vdevs. I think it's better to create encrypted files for the files that must be encrypted. I really wish you good luck on this and hope booting off the other USB drive does the trick.

P.S. Disconnect the SSD when you do this, it wasn't there before so lets remove it now. It shouldn't make a difference but it's your data and I'm sure you want it back.

 

[Jailer]

https://forums.freenas.org/index.php?threads/pool-loss-bug-11-1-rc1.59338/#post-420032

 

[joeschmuck]

https://forums.freenas.org/index.php?threads/pool-loss-bug-11-1-rc1.59338/#post-420032

OUCH! That one hurts. They should make an announcement or pull RC-1, or both. A RC should not destroy the pool.