SOLVED How to reinstall boot device and re-importing encrypted pool

[mgd]

Problem
I am going to replace my current boot device (a mirror of two USB pen drives) with two SATA SSD drives.

I have an encrypted pool (mirror of two WD RED drives) with no passphrase set, which allows the pool to be unlocked and mounted automatically when the NAS server boots. (I am aware of the dangers of having an encrypted pool and chose to have one in order to be able to RMA a faulty drive without worrying about someone being able to read my data.)

I am currently running FreeNAS 11.2-U7.

The question is how to correctly do the reinstall. I imagine this is equivalent to a situation where my USB pen drives had both crashed and the only things I had was:

• An export of the current FreeNAS configuration.
• The GELI key for the encrypted pool.

Proposed procedure
These are the steps I believe are necessary to do the migration to the new mirrored SSD boot device.

1. Boot the system using the existing USB Pen drives:
   1. Export the configuration using “System > General > Save Config”
      • Tick the checkbox “Export Password Secret Seed” if you have any passwords stored in the configuration (that could be for outgoing mail to an SMTP server).
   2. [Optional]: Take a backup of the SSH host keys in /usr/local/etc/ssh/ssh_host_*
   3. Take a backup of the GELI key used for encrypting the encrypted pool
      • either by going to “Storage > Pools”, selecting the pool, clicking the padlock icon and selecting: “Download Encrypt Key”
      • or by copying the key file directly from the boot filesystem /data/geli/<uuid>.key
      • or both – just to be sure
2. Shut down the system, unplug the storage disks (for safety so you don't accidentally install to them) and remove the old USB pen drives and store them in a safe location.
3. Do a fresh installation to the new SATA SSD boot drives.
4. Reconnect the storage disks to the system.
5. Boot the system using the new SATA SSD drives:
   1. Import the configuration that was exported. (This will reboot the system.)
   2. [Optional]: Put the SSH host keys in place on the new system
   3. Go to “Storage > Pools” where the encrypted pool should be present but locked. (We could unlock the pool here, but the key will not persist on the system.)
      1. Expand the locked pool, click the gear icon and select “Export/Disconnect”.
      2. Make sure to uncheck everything except “Confirm export/disconnect” and click “Export/Disconnect”.
      3. Click “Add”, select “Import an existing pool” and click “Next”.
      4. Select “Yes, decrypt the disks”
         1. Be careful to select both disks in the mirror,
         2. Select the previously downloaded GELI encryption key file and click “Upload”
         3. Click “Next”.
      5. Select the pool from the dropdown and click “Next”.
      6. Finally, click “Import” and the pool will be imported and the encryption key stored on the boot device.

I guess that should be it.

Is this right?
I have tried the procedure out in a VM and it seems to work.

I also tried to simply unlock the pool in step 5.3 and it works, but the GELI key is not stored on the boot device and after the next reboot the pool is locked again. The only way I could find out to put the encryption key on the boot device is by exporting and reimporting the pool so that's is the reason for all the steps under 5.3.

I would be to sad to discover that I have overlooked something and that I have lost all my data after the reinstall, so I would be really grateful if someone more knowledgeable than me could confirm whether these are the right steps.

 

[Constantin]

Have they still not fixed the boot pool bug re: supporting more than 2 drives?

Would seem to be the easiest solution: add the two new drives to the boot pool, wait a week for them to get mirrored, then use the GUI to disconnect the old USB drives.

Because of the bug, I did it one drive at a time for my starter SATADOM. Added the new satadom to the pool, let it sit for a while, then disconnected the old satadom. Then added the second new satadom. Worked just fine with my encrypted pool.

 

[mgd]

Sounds like a much simpler solution. :)
I just tested it out in a VM.

As you said, it is not possible to have a 3-way mirror, so I had to first detach one of the existing boot devices from the boot mirror. Then I added a new, larger boot device (to simulate my new SATA SSD) and checked the checkbox “Use all disk space”. When the mirror was resilvered, I removed the other “old” device and added one more new device (same size) and checked the same checkbox.

After checking that the boot mirror was online, I shutdown FreeNAS, removed the two old disks from the VM, and booted it again. It started up perfectly, and when doing a zfs list | grep freenas I can now see that the boot mirror has increased in size to take up the space of the new boot devices.

…and BTW during my testing of my complex solution above, I ran into strange situations if importing the config after importing the storage mirror, where the mirror was broken / degraded and had to be resilvered because one of the two drives was not decrypted. Not really something I want to run into with my real server, so your solution seems like a safer approach. Of course I will still backup the configuration and GELI keys, just in case. :)

 

[Constantin]

Whoo Hoo! Happiness.

 

[mgd]

@Constantin I got my two new SSD drives for the boot pool today.

• Shut down the server
• Installed both SSDs and unplugged one of the mirrored USB pendrives (to keep one if something went wrong)
• Booted – which took for ages, probably because of the USB drive being close to dying
• Attached one SSD to the boot pool and resilvered – took almost 4 hours to resilver 7.5 GB
• Removed the USB drive from the boot pool and attached the second SSD to the boot pool – resilver took 2:18 minutes

Everything was then fine and a complete boot from cold (after motherboard POST) until the system is ready now takes less than 2.5 minute.

Thanks for helping!

 

[crdouty]

Would the original procedure work for a FreeNAS Mini? The system does not have a mirrored boot drive, and I am concerned about re-importing my encrypted pool if the boot device dies. There is some confusing (to me anyway) wording in the manual about importing an encrypted pool without a passphrase.

Let's assume that I have copied off the GELI key for the pool and it has no passphrase because it contains the system dataset. Can I import this pool if I replace the boot device or mobo?

 

[Constantin]

Two thoughts:
1) I presume you have a backup in case anything does go sideways.
2) Replacing the boot pool piecemeal as described above should be risk-free.

The mini usually only comes with one SATADOM, so adding a second drive to the boot pool should be a step up re: redundancy.