SOLVED iocage fails to upgrade jail from 12.3 to 13.1 base

[FreeVel]

I have a nextcloud jail instance that I am trying to upgrade from 12.3 base image to 13.1 base image, i get the following error.

Code:

# uname -ro
FreeBSD 13.1-RELEASE
#iocage upgrade -r 13.1-RELEASE nextcloud
Mounting src into jail failed! Rolling back snapshot.

Now the catch is that, when I ran the command for the very first time, I stopped it half-way by pressing Ctrl-C as I thought it had stuck. Apparently the upgrade was going on in the background. So I might have messed up with something but I cannot tell.

I have rolled back to a snapshot that I know it works well, and repeated the process however the error keeps coming up.

Q: Any ideas on where to check for anything corrupted that stop the upgrade from happening ?

---

The jail has been created against a 12.3 base image with no issues upgrading in the past.

Code:

root@freenas:/ # iocage list
iocage list -l
+-----+-----------------+------+-------+----------+------------------+-----------------------+-----+----------+----------+
| JID |      NAME       | BOOT | STATE |   TYPE   |     RELEASE      |          IP4          | IP6 | TEMPLATE | BASEJAIL |
+=====+=================+======+=======+==========+==================+=======================+=====+==========+==========+
| 39  | nextcloud       | on   | up    | jail     | 12.3-RELEASE-p5  | vnet0|192.168.1.55/24 | -   | -        | yes      |
+-----+-----------------+------+-------+----------+------------------+-----------------------+-----+----------+----------+

Under the iocage/jails/nextcloud folder the fstab file reads likes this

Code:

# cat fstab
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/bin    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/bin    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/boot    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/boot    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/lib    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/lib    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/libexec    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/libexec    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/rescue    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/rescue    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/sbin    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/sbin    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/usr/bin    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/usr/bin    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/usr/include    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/usr/include    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/usr/lib    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/usr/lib    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/usr/libexec    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/usr/libexec    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/usr/sbin    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/usr/sbin    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/usr/share    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/usr/share    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/usr/libdata    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/usr/libdata    nullfs    ro    0    0
/mnt/MEDIA_VOL/iocage/releases/12.3-RELEASE/root/usr/lib32    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/usr/lib32    nullfs    ro    0    0
/mnt/RESILIENT_VOL/nextcloud/data/db    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/var/db/mysql    nullfs    rw    0    0
/mnt/RESILIENT_VOL/nextcloud/data/files    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/media    nullfs    rw    0    0
/mnt/RESILIENT_VOL/backup    /mnt/MEDIA_VOL/iocage/jails/nextcloud/root/mnt/backup    nullfs    ro    0    0 # Added by iocage on 2020-11-21 10:54:25

This confirms the Jail is not a clone and makes use of 12.3 base image

Code:

# zfs list -t filesystem -o name,origin -r /mnt/MEDIA_VOL/iocage/jails
NAME                                         ORIGIN
MEDIA_VOL/iocage/jails                       -
MEDIA_VOL/iocage/jails/nextcloud             -
MEDIA_VOL/iocage/jails/nextcloud/root        -

Also if I try to run the iocage upgrade command straight after the error I get the following error, apparently caused by the snapshot.

Code:

Snapshot already exists
Traceback (most recent call last):
  File "/usr/local/bin/iocage", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/iocage_cli/upgrade.py", line 40, in cli
    ioc.IOCage(jail=jail, skip_jails=skip_jails).upgrade(release)
  File "/usr/local/lib/python3.9/site-packages/iocage_lib/iocage.py", line 2086, in upgrade
    new_release = ioc_upgrade.IOCUpgrade(
  File "/usr/local/lib/python3.9/site-packages/iocage_lib/ioc_upgrade.py", line 236, in upgrade_basejail
    self.__snapshot_jail__()
  File "/usr/local/lib/python3.9/site-packages/iocage_lib/ioc_upgrade.py", line 403, in __snapshot_jail__
    ioc.IOCage(jail=self.uuid, skip_jails=True, silent=True).snapshot(name)
  File "/usr/local/lib/python3.9/site-packages/iocage_lib/iocage.py", line 1719, in snapshot
    ioc_common.logit(
  File "/usr/local/lib/python3.9/site-packages/iocage_lib/ioc_common.py", line 107, in logit
    callback(content, exception)
  File "/usr/local/lib/python3.9/site-packages/iocage_lib/ioc_common.py", line 92, in callback
    raise callback_exception(message)
iocage_lib.ioc_exceptions.Exists: Snapshot already exists

 

[FreeVel]

Upgraded from 13.0-U1 to 13.0-U1.1 and rebooted

performed

Code:

iocage upgrade -r 13.1-RELEASE nextcloud

which resulted to

Code:

nextcloud successfully upgraded from 12.3-RELEASE-p5 to 13.1-RELEASE!
Please reboot the jail and inspect.
Remove the snapshot: ioc_upgrade_2022-08-09 if everything is OK

so either the reboot or the upgrade did it, or both

 

[Patrick M. Hausen]

At your first attempt the error is right in the first line of your last iocage invocation:

Code:

Snapshot already exists

iocage creates a snapshot with a fixed name for any upgrade of a jail. When you interrupt an upgrade that snapshot remains and subsequent repeat attempts fail.

Try zfs list -t snapshot -r MEDIA_VOL/iocage/jails/nextcloud to see if that is the case and zfs destroy to remove them.

iocage error messages are a thing of beauty. Instead of catching anything that goes wrong and printing a message meaningful in the context, it just throws a python exception. Try iocage foo for fun ... shouldn't that be "foo - unknown command" or similar?