FreeNAS 11 VMs

[jamlam]

So I bit the bullet and migrated to 11 from Corral and for my use case most things are working pretty well, although I have one showstopper that I'm hoping someone can help me with...

I was running pfSense in a VM as a router using 2 vnics bridged to 2 physical nics on the server, 1 for the LAN and a dedicated WAN nic running PPPoE over a fibre modem. Trying to replicate this config in 11, I can't see any way I can set the bridged nic configuration so I just end up with 2 vnics mapped to the same physical nic and no WAN.

Does anyone know where the config for this is and if/how you can edit it?

I've also noticed that you only seem too be able to set the bootloader type to UEFI which isn't quite as big a deal but is still a pain if you're trying to move grub-based VM's from Corral :(

 

[John Rushford]

I too migrated to 11 from Corral. I lost my docker plex container but I'll be able to recover that. I was using several VM's none of which used UEFI or UEFI-CSM boot loaders. Under Corral I was using GRUB and BHYVELOAD as boot loaders so, I was unable to start any of my vms with the new 11 vm ui. I was however able to write a bash script to start up one of my vm's on my freebsd 11 nas using bhyveload and I had to manually create the bridge and tap network interfaces to get the vm on the network. I think if they add in support for the BHYVELOAD and GRUB boot loaders along with setting up bridged nics, I can then use the ui with my Corral created vm's. Choosing my zfs vm volume datasets appears to already be there.

Anyway, here are the scripted steps I used to start up one of my corral created vm's using the zvol vm images still there and BHYVELOAD boot loader:

Code:

# ifconfig tap0 create
# sysctl net.link.tap.up_on_open=1
# ifconfig bridge0 create
# ifconfig bridge0 addm igb0 addm tap0
# ifconfig bridge0 up

/usr/sbin/bhyveload -c /dev/nmdm3A -m 4096 -d /dev/zvol/zvol/vm/Susieq/os Susieq

/usr/sbin/bhyve -A -H -P -c 2 -m 4096 -s 0:0,hostbridge \
-s 1:0,ahci-hd,/dev/zvol/zvol/vm/Susieq/os \
-s 2:0,virtio-net,tap0,mac=02:a0:98:36:4d:c0 \
-s 3:0,ahci-hd,/dev/zvol/zvol/vm/Susieq/ada1 \
-s 4:0,ahci-hd,/dev/zvol/zvol/vm/Susieq/ada2 -s 31,lpc -l com1,/dev/nmdm3A -W Susieq

 

[jamlam]

Cool, thanks for that! I was thinking I had to do something in iohyve or some other utility, I'll give it a go and see if I can get things up and running.

 

[John Rushford]

Cool, thanks for that! I was thinking I had to do something in iohyve or some other utility, I'll give it a go and see if I can get things up and running.

Just a note about the devices in the code snippet. I added two additional drives to my vm later and you're likely using one. Anyway in the snippet my original vm disk is the "Susieq/os" Also, the /dev/nmdm3A is the primary side of the console. You can attach to the console on your freenas using the second half /dev/nmdmB with "cu -l /dev/nmdm3B"

 

[John Rushford]

I too migrated to 11 from Corral. I lost my docker plex container but I'll be able to recover that. I was using several VM's none of which used UEFI or UEFI-CSM boot loaders. Under Corral I was using GRUB and BHYVELOAD as boot loaders so, I was unable to start any of my vms with the new 11 vm ui. I was however able to write a bash script to start up one of my vm's on my freebsd 11 nas using bhyveload and I had to manually create the bridge and tap network interfaces to get the vm on the network. I think if they add in support for the BHYVELOAD and GRUB boot loaders along with setting up bridged nics, I can then use the ui with my Corral created vm's. Choosing my zfs vm volume datasets appears to already be there.

Anyway, here are the scripted steps I used to start up one of my corral created vm's using the zvol vm images still there and BHYVELOAD boot loader:

Code:

# ifconfig tap0 create
# sysctl net.link.tap.up_on_open=1
# ifconfig bridge0 create
# ifconfig bridge0 addm igb0 addm tap0
# ifconfig bridge0 up

/usr/sbin/bhyveload -c /dev/nmdm3A -m 4096 -d /dev/zvol/zvol/vm/Susieq/os Susieq

/usr/sbin/bhyve -A -H -P -c 2 -m 4096 -s 0:0,hostbridge \
-s 1:0,ahci-hd,/dev/zvol/zvol/vm/Susieq/os \
-s 2:0,virtio-net,tap0,mac=02:a0:98:36:4d:c0 \
-s 3:0,ahci-hd,/dev/zvol/zvol/vm/Susieq/ada1 \
-s 4:0,ahci-hd,/dev/zvol/zvol/vm/Susieq/ada2 -s 31,lpc -l com1,/dev/nmdm3A -W Susieq

I created a feature request to add BHYVELOAD and GRUB boot loader options along with nic bridge configuration. See https://bugs.freenas.org/issues/23500

 

[masterofeye]

Maybe i'm blind but where can I set the type (debian, ubuntu.. ) of the VM in the GUI?

 

[KrisBee]

Maybe i'm blind but where can I set the type (debian, ubuntu.. ) of the VM in the GUI?

In FreeNAS 11 , as of now, you can't do that. There are no templates like in FreeNAS Corral.

You can call your VM anything, it's the ISO you create it from that determines what it is.

 

[SavageAUS]

I can install an os in the vm but I've never been able to get one to boot from its hard drive. I've tried debian and Ubuntu. Is there anything extra I need to do besides click VM's and add vm, add devices, start?
My vm's always boot to efi shell.

Sent from my SM-G930F using Tapatalk

 

[KrisBee]

I can install an os in the vm but I've never been able to get one to boot from its hard drive. I've tried debian and Ubuntu. Is there anything extra I need to do besides click VM's and add vm, add devices, start?
My vm's always boot to efi shell.

Sent from my SM-G930F using Tapatalk

Short answer. Yes, you will need to carry out actions at the UEFI shell to get it to boot. After that you need to create the efi boot file that the bhyve UEFI bootrom is looking for by default on your linux VM. (Note: THe byhve UEFI bootrom has no flash memory as per a real machine, so some changes you might normally have made with real UEFI firmware do not persist.)

Long answer ... to follow if you need detailed instructions.

 

[SavageAUS]

Thanks for taking the time to answer. If you could tell me what actions I need to take at the uefi shell to get the vm's to boot I would be very appreciative.

Sent from my SM-G930F using Tapatalk

 

[SweetAndLow]

Short answer. Yes, you will need to carry out actions at the UEFI shell to get it to boot. After that you need to create the efi boot file that the bhyve UEFI bootrom is looking for by default on your linux VM. (Note: THe byhve UEFI bootrom has no flash memory as per a real machine, so some changes you might normally have made with real UEFI firmware do not persist.)

Long answer ... to follow if you need detailed instructions.

Huh? I just created a zvol, installed the os and booted it up. Tested in FreeNAS 9.10 nightly about 2 weeks ago.

Sent from my Nexus 5X using Tapatalk

 

[KrisBee]

Huh? I just created a zvol, installed the os and booted it up. Tested in FreeNAS 10 nightly about 2 weeks ago.

Sent from my Nexus 5X using Tapatalk

Which OS, and from a template or was it created from an ISO? Doubt it was either Debian or Ubuntu from an ISO , and this is in FreeNAS 11. Why else are there multiple threads asking the same question after folk are left at the UEFI shell?

 

[SavageAUS]

Huh? I just created a zvol, installed the os and booted it up. Tested in FreeNAS 10 nightly about 2 weeks ago.

Sent from my Nexus 5X using Tapatalk

We are discussing FreeNAS 11 nightlies vm's not corral. Corral was way easier for docker and vm's.

Sent from my SM-G930F using Tapatalk

 

[SweetAndLow]

Which OS, and from a template or was it created from an ISO? Doubt it was either Debian or Ubuntu form an ISO , and this is in FreeNAS 11. Why else are there multiple threads asking the same question after folk are left at the UEFI shell?

typo, i wasn't using corral i was using the 9.10 nighties. Let me try it again from a fresh install.

 

[KrisBee]

Thanks for taking the time to answer. If you could tell me what actions I need to take at the uefi shell to get the vm's to boot I would be very appreciative.

Sent from my SM-G930F using Tapatalk

It's far quicker to do than explain properly, but as it seems to be a common stumbling block and may not change once FreeNAS 11 is released. I'll attempt a quick how-to and post in this sub-forum as a separate thread ...

 

[raidflex]

So after all this is it that setup of a VM in FreeNAS 11 will not be just mounting the ISO and installing the OS as usual? Or is this strictly an issue with UEFI boot process vs legacy. Because if VM supported is to be added, it should not be half baked in and require manual boot commands just to setup one.

 

[KrisBee]

So after all this is it that setup of a VM in Freenas 11 will not be just mounting the ISO and installing the OS as usual? Or is this strictly an issue with UEFI boot process vs legacy. Because if VM supported is to be added, it should not be half baked in and require manual boot commands just to setup one.

Perhaps this is a question you should pose to the FreeNAS devs. I can only comment on the FreeNAs 11 nightly as is of today. AFAIK UEFI-CSM , what you have called legacy boot, does not work via the webui.

 

[KrisBee]

Thanks for taking the time to answer. If you could tell me what actions I need to take at the uefi shell to get the vm's to boot I would be very appreciative.

Sent from my SM-G930F using Tapatalk

It's done, see here: https://forums.freenas.org/index.php?threads/how-to-how-to-boot-linux-vms-using-uefi.54039/

Hopefully, it's error free.

 

[raidflex]

Perhaps this is a question you should pose to the FreeNAS devs. I can only comment on the FreeNAs 11 nightly as is of today. AFAIK UEFI-CSM , what you have called legacy boot, does not work via the webui.

I just hate to see FreeNAS 11 turn into another Corral. If VM support is to be added it should be done right and if its not ready for the coming release then it should not be implemented until its ready. I guess we will see when the release comes out.

 

[KrisBee]

But using UEFI as the boot method when creating a Linux guest works as you would expect it to. It's a current limitation of bhyve which uses firmware based on the OVMF tianocore project, as does Virtualbox, to support UEFI booting guests which leads to the need to use the EFI shell in some cases. I don't imagine there will be any change to this when FreeNAS 11 is released.

CentOS is one of the few distros to already have the default EFI files and fallback that the byhve UEFI firmware expects to be present . For example in CentOS & minimal:

Code:

[root@localhost efi]# pwd
/boot/efi/efi
[root@localhost efi]# cd BOOT
[root@localhost BOOT]# ls -l
total 1340
-rwx------. 1 root root 1296176 Dec  7  2015 BOOTX64.EFI
-rwx------. 1 root root  73240 Dec  7  2015 fallback.efi