I've been testing TrueNAS Scale (currently running nightly TrueNAS-SCALE-21.05-MASTER-20210504-072918) and have run into some issues with virtualization- specifically with Debian Linux (UEFI boot).
It seems that TrueNAS Scale virtualization has a similar issue as TrueNAS Core when it comes to Debian and UEFI boot (see post at: https://www.truenas.com/community/threads/bhyve-vm-uefi-boot-fails-after-each-upgrade.59825/).
Specifically, after initial install of a Debian VM (with UEFI boot) it is not able to find the boot loader (i.e. debian/grubx64.efi). This can be fixed temporarily by going into the virtual UEFI/BIOS menus of the VM and manually adding a new boot option and pointing it to grubx64.efi and then saving the changes. However, it seems rebooting and/or upgrading TrueNAS Scale will cause this saved boot setting to be lost. As a result the VM can't boot up successfully without manual intervention after a reboot of TrueNAS Scale and I've had to manually go back into the UEFI/BIOS of the VM and re-add the boot option to point it to debian/grubx64.efi.
In the TrueNAS Core post and associated bug entry https://redmine.ixsystems.com/issues/27288 a workaround is mentioned about copying debian/grubx64.efi to EFI/BOOT/bootx64.efi. I have not tried the workaround on TrueNAS scale yet to see it it works but will do so the next time I have to take my server down for maintenance or an update.
Is there anyway Scale could be updated so that it persists the UEFI/BIOS changes across TrueNAS Scale upgrades and reboots?
Doing some digging around the file system I see .fd files for each VM under /var/lib/libvirt/qemu/nvram/ (for example, /var/lib/libvirt/qemu/nvram/1_vmname_VARS.fd). This file is also referenced in the VM XML config for the vm (under /etc/libvirt/qemu/) in the <os><nvram> section. I suspect that the .fd files might be what holds the UEFI/BIOS configuration settings but I'm not 100% sure.
Thanks!
It seems that TrueNAS Scale virtualization has a similar issue as TrueNAS Core when it comes to Debian and UEFI boot (see post at: https://www.truenas.com/community/threads/bhyve-vm-uefi-boot-fails-after-each-upgrade.59825/).
Specifically, after initial install of a Debian VM (with UEFI boot) it is not able to find the boot loader (i.e. debian/grubx64.efi). This can be fixed temporarily by going into the virtual UEFI/BIOS menus of the VM and manually adding a new boot option and pointing it to grubx64.efi and then saving the changes. However, it seems rebooting and/or upgrading TrueNAS Scale will cause this saved boot setting to be lost. As a result the VM can't boot up successfully without manual intervention after a reboot of TrueNAS Scale and I've had to manually go back into the UEFI/BIOS of the VM and re-add the boot option to point it to debian/grubx64.efi.
In the TrueNAS Core post and associated bug entry https://redmine.ixsystems.com/issues/27288 a workaround is mentioned about copying debian/grubx64.efi to EFI/BOOT/bootx64.efi. I have not tried the workaround on TrueNAS scale yet to see it it works but will do so the next time I have to take my server down for maintenance or an update.
Is there anyway Scale could be updated so that it persists the UEFI/BIOS changes across TrueNAS Scale upgrades and reboots?
Doing some digging around the file system I see .fd files for each VM under /var/lib/libvirt/qemu/nvram/ (for example, /var/lib/libvirt/qemu/nvram/1_vmname_VARS.fd). This file is also referenced in the VM XML config for the vm (under /etc/libvirt/qemu/) in the <os><nvram> section. I suspect that the .fd files might be what holds the UEFI/BIOS configuration settings but I'm not 100% sure.
Thanks!