Added spare to encrypted pool. Rekey then generates a middleware/GELI error about the keyfile

ExR90

Cadet
Joined
Sep 30, 2017
Messages
7
I have a 3 vdev (each are 2 disk mirrors) encrypted pool. I added a new disk to the system. I then used the GUI to designate this new disk as a spare for the pool. That went fine.

I then went to rekey the pool (and set a passphrase), and upon doing so I got a Middleware/GELI error stating it was unable to set the passphrase on the GPTID of the spare drive (I have confirmed that the GPTID in question is the spare drive I just added), stating it could not open the keyfile. I noticed the error has the keyfile name correctly, but with .key.tmp as the file extension, instead of .key as exists on the /data/geli folder.

I have done nothing else since, no reboots, no locking of the pool either. Unclear what to do next. Yes I have backups of the pool, but it's huge so rebuilding would be last resort. Some of the other posts I found seem to indicate that spares do not get encrypted until put into action, which is fine, but unclear what to do next, if I need to do anything to avoid exploding the pool somehow should the keys be messed up.

Excerpt of the exact error below. System is Freenas 11.3-U4.1. The pool is otherwise showing healthy and the new spare is listed properly as the pool spare.

Code:
Error: Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 130, in call_method
    io_thread=False)
  File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1084, in _call
    return await methodobj(*args)
  File "/usr/local/lib/python3.7/site-packages/middlewared/schema.py", line 961, in nf
    return await f(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/pool.py", line 1475, in rekey
    await self.middleware.call('disk.geli_rekey', pool)
  File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1141, in call
    app=app, pipes=pipes, job_on_progress_cb=job_on_progress_cb, io_thread=True,
  File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1098, in _call
    return await run_method(methodobj, *args)
  File "/usr/local/lib/python3.7/site-packages/middlewared/utils/run_in_thread.py", line 10, in run_in_thread
    return await self.loop.run_in_executor(self.run_in_thread_executor, functools.partial(method, *args, **kwargs))
  File "/usr/local/lib/python3.7/site-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/disk.py", line 530, in geli_rekey
    raise CallError(f'Unable to set key: {error}')
middlewared.service_exception.CallError: [EFAULT] Unable to set key: [EFAULT] Unable to set passphrase on gptid/22717218-5061-11eb-8faa-f01fafd2b332: geli: Cannot open keyfile /data/geli/714b5528-c370-45cb-9984-a7dbde494724.key.tmp: No such file or directory.
 

ExR90

Cadet
Joined
Sep 30, 2017
Messages
7
I do have the old keys and recovery keys, as well as the new key and recovery key created during said rekey. My thought would just be to lock/unlock the pool and all would be fine, or worst case the spare is poo-poo, but before I did anything at all, I wanted to see what say you.
 
Top