SOLVED Replaced drive in encrypted pool, howto verify keys before reboot?

3nm1

Cadet
Joined
Jul 11, 2019
Messages
7
Have a bit of an language barrier issue, i followed the documentation for replacing a failed drive in my encrypted pool, after the resilvering the pool status shows as healthy, but I get stuck at this part in the documentation:

Wait until resilvering is complete before restoring the encryption keys to the pool. Restore the encryption keys before the next reboot or access to the pool will be permanently lost.

But since the only options I got are the following:
  • Create passphrase
  • Add recovery key
  • Delete recovery key
  • Encryption Rekey
  • Download Encrypt Key
Im kinda stuck on what to do. Would it be enough to press Add recovery key?

The pool was created a while ago, and i added a passphrase, and downloaded both the recovery and encryption keys, at that time,

And i downloaded the encrypt key before doing the disk swap.

Is there any way i can verify that everything is in order, before rebooting? It´s not the most valued data, and the server is just for my personal home use, but it will take a while to rip all my blue rays and dvd's again...
 
Joined
Oct 18, 2018
Messages
969
Hi @3nm1. You want to take the following steps.

  1. Add recovery key - downloads a backup key (User Key 2)
  2. Encryption Rekey - regenerates the main key (User Key 1)
  3. Create passphrase - adds passphrase to User Key 1
  4. Download Encrypt Key - downloads User Key 1

Make sure you keep the 2 downloaded files in a safe place.

Then, to verify you can lock/unlock your pool using the recovery key (User Key 2) to verify that it works.

To verify that User Key 1 works you can take the following steps assuming you're using FreeNas-11.2-U5 as your signature indicates. I assume you have only a single encrypted pool. If not there are a few additional steps.

  1. $ mv /data/geli/<key_name>.geli /data/teli/<key_name>_back.geli
  2. Reboot your computer
  3. Attempt to unlock the pool using your passphrase. It should fail
  4. Using SCP or some other tool upload User Key 1 to your FreeNAS box and put it in /data/geli/<key_name>.geli. Use the same as the original file name from step 1.
  5. Try to unlock the pool with your passphrase. It should work. If so, you now know your key worked! You can remove the extra copy $ rm /data/geli/<key_name>_back.geli

If it failed to unlock.
  1. Post the exact error messages and steps you took here
  2. Regain access to your pool by $ mv /data/geli/<key_name>_back.geli /data/geli/<key_name>.geli
  3. Unlock your pool with your passphrase
  4. repeat the step to generate User Key 1 and add the passphrase
  5. try to repeat these steps to verify
Note: I've done the above steps on my machine to learn about and verify the encryption keys. It does work, but if you make a mistake you could possibly lock yourself out of your data forever if the mistake is serious enough.
 
Last edited:

3nm1

Cadet
Joined
Jul 11, 2019
Messages
7
Hi, thanks for the reply, i get the following error message when trying to create a passphrase.
Error creating passphrase for pool MediaStorage...


Think i will just buy a temporary backup drive, and move my files to that and delete and redo the pool from scratch..
 
Joined
Oct 18, 2018
Messages
969
Did you try Encryption Rekey first?
 

3nm1

Cadet
Joined
Jul 11, 2019
Messages
7
Yes, got a "Successfully re-keyed pool MediaStorage"
 
Joined
Oct 18, 2018
Messages
969

3nm1

Cadet
Joined
Jul 11, 2019
Messages
7
That helped, doing a backup before reboot, will post my findings hopefully tomorrow. Thanks for the support.
 

3nm1

Cadet
Joined
Jul 11, 2019
Messages
7
Server rebooted and verified, thanks for the support!
 

gary_1

Explorer
Joined
Sep 26, 2017
Messages
78
For what it's worth, the documentation is rather confusing over what you've to do and why. Here's my understanding should anyone else run into this.

You have access to two keys for encryption, a recovery key (geli_recovery.key) and a main key (geli.key). The main key is the one you can create a password on.

Assuming you have unlocked your pool using the main key, then when you replace a drive, the replaced drive will work fine with the same main key as the rest of the drives. Without you doing anything else. If anything happens and you're forced to reboot, you will be able to unlock with your main key without problem. However, the new drive will _not_ have a valid recovery key (if it does have one, it certainly won't match the recovery key of the rest of the pools disks) as that key was not available to freenas at the time of the resilver.

So after resilver completes, you MUST "Add Recovery Key" to cause all disks in the pool including the newly replaced one to get a new recovery key. Download/keep this as normal and dispose of the old one as it's no longer usable.

The docs seem to get confusing as they suggest you must also rekey to generate a new main key and then create a new password, but this step is not required. It seems it's more a "best practice" than required, so that the key present (albeit it encrypted) on the disk you're disposing of, will not be usable with your pool anymore should someone manage to decrypt it.

The one part I'm _not_ sure about however, is what happens if you had unlocked your pool using the recovery key and then done the disk replace. In this instance freenas would only have access to your recovery key, hence you'd expect newly replaced disk to have a valid recovery key but in turn not to have a valid main key. In this case I'd expect you MUST "rekey" and and (optionally) "create passphrase" to ensure all disks in the pool can be unlocked by the same main key.

I've not tested that however and if any of the above is not actually correct, then I'll once more state the docs are unclear as to what and more importantly why, you need to do what it states.
 

hungarianhc

Patron
Joined
Mar 11, 2014
Messages
234
Okay. This is hilarious. I came to the forums tonight with detailed questions... I just replaced a drive in an encrypted pool, and I followed the documentation AND I read around the FreeNAS forums, and I THINK I did it right... but I wanted to verify here. Now I come here, and I see that someone had literally my exact same question. I feel like the documentation could be improved by just putting this into a few basic steps. I'll file a bug and report back.

EDIT: To make matters even more confusing...

1) When you click "Add Encrypt Key" it DOWNLOADS a file, and the name of the file is geli_recovery.key
2) When you click "Download Encrypt Key" a dialog box pops up asking if you want to download a RECOVERY key, but the name of the file is geli.key

Could it be possible to make it even more confusing?!?!
 
Last edited:

gary_1

Explorer
Joined
Sep 26, 2017
Messages
78
The popups are not just confusing, they're wrong.

When people replace a drive in an encrypted pool it should warn you that you have to add a new recovery key as the existing recovery key will be invalidated. Ideally a note/aside section of the docs should explain that it is because the recovery key isn't available during a rekey if you unlock with your main key, so whilst the old drives in the pool could still be unlocked using one, the new drive will not. Hence replacing the recovery key to ensure all drives can be unlocked with the same key.

The download encrypt key button should not mention recovery key at all as it's dealing with the main key. As you note, it does download the correct key, it's just the button label and text is completely wrong.

We know how freenas works if you unlocked the pool with the main key and replace a drive, so could adjust the docs to make that clearer, even if it's just adding some explination as an aside on how the encryption works, how the main key will work on old and new drives, but recommendation is to rekey (due to exposure of old encrypted keys on the dead drive) and that the new drive will be missing a recovery key unless you "add recovery key" to give the entire pool a new recovery key.

That said, before that happens, someone with a VM or that knows the source, could do with checking, is it possible after unlocking a pool with a recovery key, to replace a drive? You'd expect a pool unlocked with a recovery key to mean the new drive will have its master key encrypted with a valid recovery key but since there was no main key available, that slot would be invalid, i.e the exact reverse of the normal situation where people have pools unlocked with the main key. But, does freenas do that?
 
Last edited:

hungarianhc

Patron
Joined
Mar 11, 2014
Messages
234
Hey all,

I filed a bug here. I think there are really two bugs to be filed - a documentation bug (what I filed) and a bug around changing the text descriptions of the things you need to click. I felt the documentation was a bigger issue so I started there.
 
Top