[EFAULT] Pool could not be imported: 1 devices failed to decrypt.

[Papid1975]

One of my pools is shown “OFFLINE” in the dashboard after boot. It says “Data not available while pool is encrypted.”

Going to ”System” → “Pools” it is listed as “LOCKED”. There is a little padlock symbol. When I click it, it offers my to unlock the pool: “Unlock the pool with either a passphrase or a recovery key.”.

I used both, first the passphrase, then the recovery key file. In both cases I get this message:

[EFAULT] Pool could not be imported: 1 devices failed to decrypt.

What is wrong?

 

[Papid1975]

Also…Hi! :)

 

[Samuel Tai]

What's the layout of your pool, and what version of FreeNAS?

 

[Papid1975]

It is just one disk. I am using FreeNAS-11.3-U2.

 

[Samuel Tai]

Do you have other encrypted pools? Or is this your only pool?

 

[Papid1975]

I boot from a USB thumb drive. Main storage is a mirror with two hard drives (this pool is working fine). I mounted the now locked single-disk pool as my home folder. So, the System boots up, I can unlock my main storage. I just can’t access my home folder anymore.

 

[Samuel Tai]

Did you have different passphrases and recovery keys for your home pool and your main pool? That's what's likely going on. You're using the wrong pool's passphrase or recovery key to attempt unlocking the home pool.

 

[Papid1975]

I thoroughly stored the different recovery keys when I encrypted both pools. I first set up the mirrored pool, saved the key with the pools name into an encrypted container on another machine. After that happened I encrypted the second drive, then named the recovery key and put it into its secure archive. I am 100% sure that I used the correct key.

I am not totally sure but I think that I couldn’t unlock the pool after the update from 11.2 to 11.3. Maybe the unlock didn’t work before as well (and I didn’t realize it).

Can this also be a ZFS file system fault? Or would that have different symptoms?

How would I try to debug this on the command line? I am new to ZFS.

 

[Samuel Tai]

Ouch. Is your home pool where you stored the system dataset? This is a known issue with the 11.2->11.3 upgrade. Try rebooting back into 11.2, and removing the passphrase from the home pool.

READ THIS before you upgrade to 11.3, if you have GELI-encrypted pools!

This is in the Known Impacts section of the Release Notes at https://www.ixsystems.com/blog/library/freenas-11-3-release/. The system no longer allows moving the system dataset to an encrypted pool containing a passphrase. Since Directory Services and some SMB state information is stored in the...

www.ixsystems.com

 

[Papid1975]

That sounds like a promising idea. Thank you very much!

I guess I can’t downgrade the system I am running now?

I don’t have physical access to the machine, but that is what I need, right? Another USB drive with FreeNAS 10.2 on it?

 

[Samuel Tai]

No, your 11.2 installation is still there. Look under System->Boot. Set it to be active at the next boot.

Alternatively, at the boot menu, you can select the previous boot environment with option 4.

Finally, you could also do it from the shell via bectl list to list the available boot environments, and then bectl activate <name of 11.2 boot environment>.

 

[Papid1975]

Uh, interesting. Never saw this. Will try that in a few hours. Do I need to set this keep flag for my current environment (all the environments listed are set to “no”)?
Thank you so much, really hope this does the trick. And if it doesn’t I hope you’re still around with a new ides ;) Thanks again!

 

[Samuel Tai]

Keep is just to protect the environment from auto-pruning. You only need to set the next flag to make it active on the next boot.

 

[Papid1975]

Rebooted to FreeNAS 11.2-U8 now, same problem. Slightly different error message when trying to unlock (through passphrase or recovery key) though:

[MiddlewareError: Volume could not be imported: 1 devices failed to decrypt]

Does that mean, the 11.2→11.3 update is not the problem here?

 

[Samuel Tai]

No, it means your passphrase and recovery key are no longer valid. Your only hope is to try the keys in /data/geli. If none of them work, then your pool is lost.

If one of those keys do work, then you should immediately:

1. Reset keys on that pool.
2. Don't set a passphrase when prompted.
3. Save a new recovery key on that pool.
4. Remove the existing 11.3 boot environment, and update to 11.3 again from System->Update.

 

[Papid1975]

No key from /data/geli works (there now is a different error message again: “Error Unlocking”).

I guess, I’ll start accepting my data is lost. Anyways, I still don’t fully get what happened. Why are passphrase/key for the mirrored pool still valid but the ones for the “home pool” is not valid after the update? From my perspective the only difference between the pools is the fact that I stored the system dataset on the “home pool”.

I also realised that geom disk list does not list that locked “home pool” drive. Might the drive just be broken or disconnected after all (can’t check it as I don’t have physical access)?

 

[Samuel Tai]

Yes, unfortunately, the 11.3 upgrade does bad things to encrypted datasets hosting the system dataset, unless you took the precaution of reading the release notes ahead of time and understood you needed to remove the passphrase.

Path to success for system upgrades

Path to success for system upgrades Too many folks here just blindly upgrade, and find themselves in a pickle afterwards (jails no longer working, encrypted pools not unlocking, or other nastiness). All too often, their predicaments are addressed in the upgrade's release notes, or in the Guide...

www.ixsystems.com

Once you do get physical access back, you could try reconnecting the drive, to see if it comes back in geom disk list. This is a very long shot, but some members here have been able to revive their encrypted pools this way, and their old passphrases/recovery keys pre-upgrade still worked.

 

[Papid1975]

Alright, thank you so much for your time. This has been very insightful. Also a good reminder why it is important to read release notes before upgrading ;)