fuzzydunlop
Cadet
- Joined
- Dec 12, 2023
- Messages
- 3
Hello, I have a server running TrueNAS 13 which has been running with two usb drives for the boot-pool and a single storage pool of two 4tb drives in a mirror vdev for many years. This pool is geli encrypted as it was originally created when the server was built, running FreeNAS 11, but the server has since been upgraded all the way to TrueNAS (13.0 U6). This is my drive setup, the bottom entry is not relevant, it's just a flash drive plugged into the host:
I then used the menu under Storage > Pools > Gear Icon > Add vdevs. This appeared to work as expected (and I downloaded the new encryption key when prompted), but eventually the server was rebooted and now the entire pool fails to decrypt. When I go to Storage > Pools I see this:
When I look for geli in console.log I see several entries which seem to indicate either the geli provider or the key file are incorrect
The key still exists in the proper folder
The geli list still shows two providers as expected, but they appear to have different geom IDs than TrueNAS is expecting
I have not yet done anything more on this host, as I wanted to check here first, but I was able to recreate the issue in a VM so I could test potential solutions by doing the following:
In my VM lab environment I've tried the following with no success:
Does anyone have suggestions on other options to try, further debugging steps, or good reference materials to read through? I've read the other forum posts where users report similar errors decrypting geli pools, but most seem to be resolved by placing the key in the right location which didn't work here. I do have a cloud backup of the important data on this drive, but restoring that data would be costly in egress fees and require me to do a decent amount of reconfiguring on this server so I'd prefer to be able to decrypt this existing pool if possible.
I then used the menu under Storage > Pools > Gear Icon > Add vdevs. This appeared to work as expected (and I downloaded the new encryption key when prompted), but eventually the server was rebooted and now the entire pool fails to decrypt. When I go to Storage > Pools I see this:
When I look for geli in console.log I see several entries which seem to indicate either the geli provider or the key file are incorrect
The key still exists in the proper folder
The geli list still shows two providers as expected, but they appear to have different geom IDs than TrueNAS is expecting
I have not yet done anything more on this host, as I wanted to check here first, but I was able to recreate the issue in a VM so I could test potential solutions by doing the following:
- Install FreeNAS 11
- Create encrypted pool of one mirror vdev (uses geli)
- Upgrade to TrueNAS 12 and then to TrueNAS 13.0 u6
- Add new mirror vdev to pool as described above above
- Reboot, now I see the pool failed to decrypt
In my VM lab environment I've tried the following with no success:
- Run with this key but the proper geom namesCode:
geli attach
-
- Run the same but with the encryption key I downloaded when adding the new vdev, same results as above
- Export/disconnect pool in UI using the gear icon menu so I can try to reimport it, fails to export
-
- Considered using but there's only the boot pool available when runningCode:
zpool export
Code:zpool list
-
- Try using , which works (they no longer show when listing geoms) but the pool still shows up in the UI even after rebooting and the drives are not available in the dropdown when trying to add a new pool.Code:
geli detach
- Also tried to after this, both with the key originally on the server filesystem as well as the key downloaded when adding a vdev, but in both cases I see the same "error with at least one provider" shown aboveCode:
geli attach
- Also tried to
Does anyone have suggestions on other options to try, further debugging steps, or good reference materials to read through? I've read the other forum posts where users report similar errors decrypting geli pools, but most seem to be resolved by placing the key in the right location which didn't work here. I do have a cloud backup of the important data on this drive, but restoring that data would be costly in egress fees and require me to do a decent amount of reconfiguring on this server so I'd prefer to be able to decrypt this existing pool if possible.