Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

[HOWTO] How-to Boot Linux VMs using UEFI

dtom10

Member
Joined
Oct 16, 2014
Messages
81
It's not M$ doing this. The issue boils down to secureboot. I think that Centos expects a secureboot enabled UEFI which bhyve is not so it failsback to legacy bios for the bootloader which fails in bhyve since it always presents an UEFI bios no matter if it's UEFI or UEFI-CSM.

I'm not sure if my assumption is right, I'm not that low-level but I got it working, paste data and I'll work with you to get it working if you still have issues.
 

Binary Buddha

Member
Joined
Mar 6, 2016
Messages
126
It's not M$ doing this. The issue boils down to secureboot. I think that Centos expects a secureboot enabled UEFI which bhyve is not so it failsback to legacy bios for the bootloader which fails in bhyve since it always presents an UEFI bios no matter if it's UEFI or UEFI-CSM.

I'm not sure if my assumption is right, I'm not that low-level but I got it working, paste data and I'll work with you to get it working if you still have issues.
Yes and no. You are correct. It'd due to mokutil and shim not making the EFI directories because CentOS thinks it's on a BIOS system. However according to the bug report they're waiting for Microsoft to sign the shim to correct that issue before they can update the repo. Yeah... my FSF fist is shaking about that.

I was able to install it using CentOS 1503-1 and then blacklist mokutil and shim from updates and it works and is more or less up-to-date.

=== Not working MOKUTIL ===
Name : mokutil
Version : 12
Release : 1.el7.centos
=== Working MOKUTIL===
Name : mokutil
Version : 0.7
Release : 5.el7
=== Not working SHIM ===
Name : shim-x64
Version : 12
Release : 1.el7.centos
=== Working SHIM===
Name : shim
Version : 0.7
Release : 5.2.el7.centos.2

As @KrisBee pointed out; https://bugs.centos.org/view.php?id=15522. There's an unsigned RPM there as well if you want to play around with it.

I may be wrong. But after 1503-1 I think they use the new MOKUTIL and SHIM which would fail the installs. So, I'm wondering if I should wait until they update the install ISOs or see if I can throw together a Kickstart image that would work and have the updated "working" software.
 

dtom10

Member
Joined
Oct 16, 2014
Messages
81
I didn't go through that much trouble, the link in my post provides several options to fix it which worked for me. Just pave all the files. one thing I didn't mention is that I manually partitioned my VM drive and created a /boot/efi partition.

The default for any redhat clones is lvm. You can make a /boot partition and the rest as lvm if you really want to.
I've created mine as plain partitions and xfs instead. I've done an update after installing excluding the mokutil and shim packages in yum config and works fine.
 
Last edited:

Binary Buddha

Member
Joined
Mar 6, 2016
Messages
126
I didn't go through that much trouble, the link in my post provides several options to fix it which worked for me. Just pave all the files. one thing I didn't mention is that I manually partitioned my VM drive and created a /boot/efi partition.

The default for any redhat clones is lvm. You can make a /boot partition and the rest as lvm if you really want to.
I've created mine as plain partitions and xfs instead. I've done an update after installing excluding the mokutil and shim packages in yum config and works fine.
So yes, normally /boot is the first partition and the second is the LVM partition. I'm going to try and see if I can setup a Kickstart POST script to create the directories and copy the EFI over. I think it's essentially crapping out because it's not finding the EFI where it expects it. I'm assuming /boot/efi doesn't need it's own partition separate from /boot. I normally do the STIG partitioning scheme of creating partitions for /boot, /var, /var/log/audit, /tmp, /home, and swap. All except /boot are LVs.
 

Patrick M. Hausen

Dedicated Sage
Joined
Nov 25, 2013
Messages
1,588
Hey guys,

just wanted to inform everyone who has not yet switched to the latest Ubuntu LTS that 18.04 LTS installs and boots just fine with bhyve and FreeNAS 11.2. VirtIO disk and network - no special procedures necessary.

Kind regards,
Patrick.
 

kifeo

Newbie
Joined
Mar 6, 2019
Messages
1
I still have the issue with debian 9.8 :
boot from CD, install the system, when installed, shut down, delete the CDROM device (as I could not find how to eject the disk), restart, and there I have the shell.
Freenas is 11.2-STABLE
 

Patrick M. Hausen

Dedicated Sage
Joined
Nov 25, 2013
Messages
1,588
In that case you have the known issue that there is no default boot loader in /boot/efi/EFI/BOOT. Installing rEFInd is one option to fix this. Yo need to map the ZVOL into another VM, possibly FreeBSD, that boots fine, then place the BOOTX64.EFI file in the aforementioned directory. Or boot from CD again if that offers a "repair mode" besides installation.

HTH,
Patrick
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,082
I still have the issue with debian 9.8 :
boot from CD, install the system, when installed, shut down, delete the CDROM device (as I could not find how to eject the disk), restart, and there I have the shell.
Freenas is 11.2-STABLE
Unlike say later versions of Ubuntu. Debian linux does not "Force grub-efi installation to the removable media path" by default (see: https://wiki.debian.org/UEFI#Force_grub-efi_installation_to_the_removable_media_path ). It can done during the install if you use expert mode, or post-install by using the command shown in #146 above.

Currently, FreeNAS does not emulate ejecting a disk from a cdrom device. Removing a CDROM device after a VM install can cause your to lose network connectivity if the bhyve slot number the virutal NIC is attached to changes. This in turn changes the "Predictable Network Interface Name" inside the VM.
 

Patrick M. Hausen

Dedicated Sage
Joined
Nov 25, 2013
Messages
1,588
Unlike say later versions of Ubuntu. Debian linux does not "Force grub-efi installation to the removable media path" by default (see: https://wiki.debian.org/UEFI#Force_grub-efi_installation_to_the_removable_media_path ). It can done during the install if you use expert mode, or post-install by using the command shown in #146 above.
Thanks for that - I somehow missed it. So it seems like rEFInd is not needed anymore outside of physical multi boot environments where it probably still serves a good purpose.
Currently, FreeNAS does not emulate ejecting a disk from a cdrom device. Removing a CDROM device after a VM install can cause your to lose network connectivity if the bhyve slot number the virutal NIC is attached to changes. This in turn changes the "Predictable Network Interface Name" inside the VM.
BTDT - enp0s5 to enp0s4, whatever that is supposed to mean.

Kind regards,
Patrick
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,082
It will be interesting to see of the policy has changed in the forthcomig debian 10 installer. Linux using "Predictable Network Interface Name" is a nuisance for Byhve VMs and something the devs seem not to have accounted for. Your example is typical , the VM detects the NIC has moved from pci slot 4 to 5 when the CDROM device is removed. You can see this changing in bhyve but just tailing /var/log/middelwared.log
 

eseanq

Neophyte
Joined
Mar 25, 2019
Messages
6
Just came across this thread while trying to do a fresh install for a Mint VM. How do I go about selecting the grubx64.efi file since its a fresh install?
 

diskdiddler

Dedicated Sage
Joined
Jul 9, 2014
Messages
2,107
I can't get the OS to even boot the ISO, to install the OS, let alone re-configure the installed OS on the virtual disk to continue to boot?

How are people managing to boot the ISO files?
 

diskdiddler

Dedicated Sage
Joined
Jul 9, 2014
Messages
2,107
It will be interesting to see of the policy has changed in the forthcomig debian 10 installer. Linux using "Predictable Network Interface Name" is a nuisance for Byhve VMs and something the devs seem not to have accounted for. Your example is typical , the VM detects the NIC has moved from pci slot 4 to 5 when the CDROM device is removed. You can see this changing in bhyve but just tailing /var/log/middelwared.log
I am attempting Debian 10 net installer and I simply can not boot the ISO.
https://i.imgur.com/SLOuAdd.png
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,082
I am attempting Debian 10 net installer and I simply can not boot the ISO.
https://i.imgur.com/SLOuAdd.png
Deb10 installs for me. Usual caveats re: using advanced options > expert install and answering yes to install grub to EFI removable path ... Pre install set VNC res to no higher than 800x600 and booting after removal of cd-rom device requires editing of /etc/network/interfaces to restore network connectivity.
 

Attachments

diskdiddler

Dedicated Sage
Joined
Jul 9, 2014
Messages
2,107
Deb10 installs for me. Usual caveats re: using advanced options > expert install and answering yes to install grub to EFI removable path ... Pre install set VNC res to no higher than 800x600 and booting after removal of cd-rom device requires editing of /etc/network/interfaces to restore network connectivity.
Ok sorry but I have to know, what version of FreeNAS are you using.
What exact ISO for debian did you use.
Which precise settings did you choose for the VM itself

and finally, did you need to mess with the UEFI menu thing where it skips startup.nsh and you had to type "exist" mess in the menus to get it to boot or *it just outright booted your iso* without needing to mess with the options.?


My specs are in my reply, I literally can't get the ISO to load.
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,082
FN version as per signature.
debian-10.0.0-amd64-netinst.iso

Settings noting special except pre-install set VNC resolution to 800x600 and check wait for VNC connection, virtio for disk and NIC, memory 1024MB. and see my previous comments. You shouldn't see the EFI shell as your screen cap shows. Are you re-using a zvol that's had linux on before? Is your ISO good?
 

diskdiddler

Dedicated Sage
Joined
Jul 9, 2014
Messages
2,107
Embarassing, switching to the AMD64 iso (god knows how) I had the i386 32bit. Fixed the issue - perhaps installing Debian may result in the bootloader being stuffed but the fix for that, from these threads sounds semi-reasonable.

I am still utterly baffled by DietPi - what do you think the best method of getting that on my machine is? I don't suppose you could RAR (best compression) the RAW file you wrote out of DietPi and put it online somewhere so I can DD it into a (new ZVOL?...) on my machine

(I still had boot issues from the DietPI ISO file mind you, not just the IMG file)
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,082
@diskdiddler For DietPi, simplest route is to use the UEFI iso to install in virtualbox on a linux/Windows desktop/laptop first, then go from there ...

Trying to upload a 10GB RAW file is not practical for me in the UK with < 1mb/s upload internet speed.
 

diskdiddler

Dedicated Sage
Joined
Jul 9, 2014
Messages
2,107
As a fellow 1mbit upload guy, I can sympathize.



Mind you I expect 90% of it to be 0 and the file to be a similar 400mb as the original download from them.


Regardless I'll try to set up virtual box on my laptop.

Would I be right in assuming the destination zvol I create on the FreeNAS server should be the same size or slightly larger?

Let me know your zvol name and EXACT DD command and I'll adapt it to my system.
 

KrisBee

Neophyte Sage
Joined
Mar 20, 2017
Messages
1,082
Actually, I'd forgotten, I 'd already wiped all the dietpi stuff last night as I had no use for it.

A couple of things about installing DietPi in vbox. Depending what software you choose to install inside DieptPi, you probably can get away with 4 to 8GB for the vdi disk size. Make sure you enable EFI in the virtual machine setting ( System > motherboard). This is essential for it to work in bhyve using the UEFI boot method. You will drop to an EFI shell in Vobx when you start the DietPi VM. Just navigate to "Boot Maintenance Manager" then to "Boot from File" then <EFI>, <debian> and pick the shim efi file to boot.

Lastly, to avoid needing to fix EFI booting in bhyve, apply the same fix as you would in bhyve to the virtualbox DietPi VM. Use this command as root in a DietPi: grub-install --efi-directory=/boot/efi --boot-directory=/boot --removable

After setting up DietPi as you wish in Vbox, Close the VM and convert the vdi to a raw file using the VBoxManage clonehd command. Copy the raw file to FN. with scp/winscp for example, to a place of your choice.

Create zvol on FN of same size, or slightly larger, as the raw. You can use the GUI or CL to do that. The dd command is of the simple format:

dd if=<path to raw file> of=<path to zvol>

The raw file path will begin with /mnt<your pool name/ ...

The path to your zvol will begin with /dev/zvol/<your pool name>/ ......
 
Top