Virtualbox jail not running after virtualbox-ose-kmod updated

[amires]

how do you do the --ignore-missing parameter

pkg add libvncserver-0.9.9_5.txz

pkg add --ignore-missing libvncserver-0.9.9_5.txz

 

[Tsaukpaetra]

Am I assuming correctly that nobody has the right packages for FreeNAS 11?

I'm stuck between three rocks and holding on because after moving to bhyve I discovered apparently it's not working very well at all on my production server (despite being fine on the dev box).

 

[stillka]

I have configured new jail in Freenas 11 but same error:

The virtual machine 'WindowsXP' has terminated unexpectedly during startup with exit code 1 (0x1)

any idea whats wrong?

root@virtualbox:/ # pkg info |grep virtualbox
phpvirtualbox-5.0.5 AJAX Web Interface for VirtualBox
virtualbox-ose-5.1.22 General-purpose full virtualizer for x86 hardware
virtualbox-ose-kmod-5.1.22 VirtualBox kernel module for FreeBSD

 

[SaskiFX]

Same problem with mine. I got it working on 9.10.2-U3 but I just jumped to 11-STABLE and it's failing.

 

[Kam]

virtualbox-ose-kmod should be present also under freenas host. in 9.10.x it is, in 11.0 it is missing. See:

Feature #23926: VirtualBox kernel modules missing in FreeNAS 11

 

[stillka]

in FreeNAS 11 RC3 is kernel module back but I have no success to make Virtualbox jail running...anybody succeeded?

root@freenas:~ # pkg info | grep -i virtualbox
virtualbox-ose-kmod-5.1.6 VirtualBox kernel module for FreeBSD

root@freenas:~ # kldload vboxdrv
root@freenas:~ # kldstat -v | grep box
30 1 0xffffffff82cb7000 836dd vboxdrv.ko (/boot/modules/vboxdrv.ko)
498 vboxdrv

in the jail:
The virtual machine 'WindowsXP' has terminated unexpectedly during startup with exit code 1 (0x1)

dmesg:
SUP_IOCTL_COOKIE: Version mismatch. Requested: 0x280000 Min: 0x280000 Current: 0x260000


The module is for Virtualbox 5.1.6 but 5.1.22 exist in the repository.....

 

[Kam]

I had no time to test it yet, but do you have vbox drivers loaded?
check "kldstat". There should be 3 "vbox" drivers.
If not, add folowing Tunables:

type "Loader", vboxdrv_load YES
type "rc.conf", vboxnet_enable YES

but I don't know where to get virtualbox 5.1.6 for freebsd 11.
can I use my old jail based on freebsd 10 in freebsd 11 host?

 

[stillka]

Could somebody prepare a correct Virtualbox jail template which can be used?

 

[Pseudobolt]

Hi all,

for those that are still looking for a solution, here is what worked for me.

Warning: this involves updating the vbox kernel driver on your FreeNAS host. This is definitely unsupported and might cause unforeseen problems in future, eg with FreeNAS upgrades. (Or it might not -- I don't know.)

1) Update the file /usr/local/etc/pkg/repos/FreeBSD.conf to read as follows -- you'll have to add the URL line and change the second line to 'enabled: yes':

FreeBSD: {
url: "pkg+http://pkg.FreeBSD.org/freebsd:10:x86:64/quarterly",
enabled: yes
}

2) Upgrade the pkg tool, since it will insist you do this before anything else anyway:
pkg install --repository FreeBSD pkg

3) Install the newer vbox-ose-kmod. NOTE: Before you enter "y" to the prompt, you might want to check that the version it wants to install matches the virtualbox version in your jail -- if it doesn't match then your VMs still won't run; you should check that you have the same url in your repo config in the jail as you have just entered above.
pkg install --repository FreeBSD virtualbox-ose-kmod

4) Reboot the FreeNAS server. You can probably also use kldunload / kldload to load the new driver versions without rebooting, but I haven't tested it that way (if you do this, make sure you get them all, as I think there's more than one vbox driver module you'd have to unload / reload if you do it that way).

Note that if you want to do this again in future (maybe you update your jail and need to update the host too, or it gets blown away by a FreeNAS upgrade), you will probably have to edit the .../repos/FreeBSD.conf file again, since this file apparently gets reset to default on boot.

 

[Kam]

Hi,
is it a good idea to install latest kmod drivers into freenas host? latest virtualbos-ose-kmod version was compiled with the latest depended packages. and also latest kernel, to which it is integrated as it runs as a kernel module.

I was working on other solution - to compile the same version of virtualbox-ose in a jail, as it is now installed in Freenas 11.0-RC4 system. Currently it is virtualbox-ose-kmod 5.1.6, which matches revision 423390 of 5.1.6_1 version of virtualbox-ose.

Here are the steps and shell script you can use for automated build in a jail. Just create or copy vboxcreate.sh script into new 11.0 jail and execute it. Take a coffe, complilation takes more hours..

Steps to create new virtualbox jail
===================================
Jails -> Add Jail -> "virtualbox"

jexec virtualbox

vi vboxcreate.sh

#!/bin/sh
echo Installing virtualbox
rm -rf /usr/ports
# workaround for svn problem without ipv6 connectivity to use ipv4 svn.freebsd.org address
host svn.freebsd.org | awk '/has address/ { print $4 " svn.freebsd.org"; exit }' >> /etc/hosts
svn checkout --non-interactive --trust-server-cert -r 423390 https://svn.freebsd.org/ports/head /usr/ports
sed -i -e '/svn.freebsd.org/d' /etc/hosts
export BATCH=yes
# compilation of virtualbox-ose fails without bulding python3 port first
cd /usr/ports/lang/python3 && make install clean
cd /usr/ports/emulators/virtualbox-ose && make install clean
pkg -o ASSUME_ALWAYS_YES=YES install phpvirtualbox apache24 mod_php56
pkg -o ASSUME_ALWAYS_YES=YES clean
pw groupadd -n vbox -g 1001
echo "VBox123" | pw useradd -n vbox -c vbox -u 1001 -s /bin/sh -m -g vbox -G vboxusers -d /home/vbox -h -
sed -i -e 's/'\''pass'\'';/'\''VBox123'\'';/' /usr/local/www/phpvirtualbox/config.php
sed -i -e 's/apache24\/data/phpvirtualbox/g' /usr/local/etc/apache24/httpd.conf
sed -i -e '/AddType application\/x-gzip \.gz \.tgz/a\
\
AddType application\/x-httpd-php .php\
AddType application\/x-httpd-php-sources .phps\
' /usr/local/etc/apache24/httpd.conf
sed -i -e 's/#ServerName www.example.com:80/ServerName 127.0.0.1:80/' /usr/local/etc/apache24/httpd.conf
sed -i -e 's/@constant('\''PHPVBOX_VER'\'')/"5.1-0"/' /usr/local/www/phpvirtualbox/endpoints/api.php
sysrc apache24_enable="YES"
sysrc vboxwebsrv_user="vbox"
sysrc vboxwebsrv_enable="YES"
service vboxwebsrv start
service apache24 start
echo Done. Connect to phpvirtualbox at http://`ifconfig -a | grep inet | grep -v inet6 | grep -v 127.0.0.1 | awk '{print $2}'` with user "admin" password "admin"

sh vboxcreate.sh

In FreeNAS web interface
========================
Add Tunable, type "Loader":
vboxdrv_load="YES"
Add Tunable, type "rc.conf":
vboxnet_enable="YES"

and reboot to load vbox drivers.

I haven't tested it fully yet, as I don't have Freenas 11 on physical machine. Only running under other virtualbox and as it does not allow nested virtualisation, I cannot check it if it really works.

Please report back if you try it if it works or you suggest some changes.

Enjoy!

 

[rglover666]

I tried this and it does allow me to bring up phpvirtualbox and add an existing image and start it successfully. The issue that I have now is that my USB devices do not work in this image any longer (they worked fine using 9.10.1-U4.) Is there anything I can do to fix this? I have seen other posts that state that USB support does not work in 5.1.6 but I was hoping that this was the reason for the 5.1.6_1 version.

Hi,
is it a good idea to install latest kmod drivers into freenas host? latest virtualbos-ose-kmod version was compiled with the latest depended packages. and also latest kernel, to which it is integrated as it runs as a kernel module.

I was working on other solution - to compile the same version of virtualbox-ose in a jail, as it is now installed in Freenas 11.0-RC4 system. Currently it is virtualbox-ose-kmod 5.1.6, which matches revision 423390 of 5.1.6_1 version of virtualbox-ose.

Here are the steps and shell script you can use for automated build in a jail. Just create or copy vboxcreate.sh script into new 11.0 jail and execute it. Take a coffe, complilation takes more hours..

Steps to create new virtualbox jail
===================================
Jails -> Add Jail -> "virtualbox"

jexec virtualbox

vi vboxcreate.sh

#!/bin/sh
echo Installing virtualbox
rm -rf /usr/ports
# workaround for svn problem without ipv6 connectivity to use ipv4 svn.freebsd.org address
host svn.freebsd.org | awk '/has address/ { print $4 " svn.freebsd.org"; exit }' >> /etc/hosts
svn checkout --non-interactive --trust-server-cert -r 423390 https://svn.freebsd.org/ports/head /usr/ports
sed -i -e '/svn.freebsd.org/d' /etc/hosts
export BATCH=yes
# compilation of virtualbox-ose fails without bulding python3 port first
cd /usr/ports/lang/python3 && make install clean
cd /usr/ports/emulators/virtualbox-ose && make install clean
pkg -o ASSUME_ALWAYS_YES=YES install phpvirtualbox apache24 mod_php56
pkg -o ASSUME_ALWAYS_YES=YES clean
pw groupadd -n vbox -g 1001
echo "VBox123" | pw useradd -n vbox -c vbox -u 1001 -s /bin/sh -m -g vbox -G vboxusers -d /home/vbox -h -
sed -i -e 's/'\''pass'\'';/'\''VBox123'\'';/' /usr/local/www/phpvirtualbox/config.php
sed -i -e 's/apache24\/data/phpvirtualbox/g' /usr/local/etc/apache24/httpd.conf
sed -i -e '/AddType application\/x-gzip \.gz \.tgz/a\
\
AddType application\/x-httpd-php .php\
AddType application\/x-httpd-php-sources .phps\
' /usr/local/etc/apache24/httpd.conf
sed -i -e 's/#ServerName www.example.com:80/ServerName 127.0.0.1:80/' /usr/local/etc/apache24/httpd.conf
sed -i -e 's/@constant('\''PHPVBOX_VER'\'')/"5.1-0"/' /usr/local/www/phpvirtualbox/endpoints/api.php
sysrc apache24_enable="YES"
sysrc vboxwebsrv_user="vbox"
sysrc vboxwebsrv_enable="YES"
service vboxwebsrv start
service apache24 start
echo Done. Connect to phpvirtualbox at http://`ifconfig -a | grep inet | grep -v inet6 | grep -v 127.0.0.1 | awk '{print $2}'` with user "admin" password "admin"

sh vboxcreate.sh

In FreeNAS web interface
========================
Add Tunable, type "Loader":
vboxdrv_load="YES"
Add Tunable, type "rc.conf":
vboxnet_enable="YES"

and reboot to load vbox drivers.

I haven't tested it fully yet, as I don't have Freenas 11 on physical machine. Only running under other virtualbox and as it does not allow nested virtualisation, I cannot check it if it really works.

Please report back if you try it if it works or you suggest some changes.

Enjoy!

 

[rglover666]

I fixed my USB problem myself by separating the svn checkout from the build process and manually merging the changes mentioned in this link https://svnweb.freebsd.org/ports?view=revision&revision=432460 into the files before running the build process.

 

[rdmitry0911]

Hi,
is it a good idea to install latest kmod drivers into freenas host? latest virtualbos-ose-kmod version was compiled with the latest depended packages. and also latest kernel, to which it is integrated as it runs as a kernel module.

I was working on other solution - to compile the same version of virtualbox-ose in a jail, as it is now installed in Freenas 11.0-RC4 system. Currently it is virtualbox-ose-kmod 5.1.6, which matches revision 423390 of 5.1.6_1 version of virtualbox-ose.

Here are the steps and shell script you can use for automated build in a jail. Just create or copy vboxcreate.sh script into new 11.0 jail and execute it. Take a coffe, complilation takes more hours..

Steps to create new virtualbox jail
===================================
Jails -> Add Jail -> "virtualbox"

jexec virtualbox

vi vboxcreate.sh

#!/bin/sh
echo Installing virtualbox
rm -rf /usr/ports
# workaround for svn problem without ipv6 connectivity to use ipv4 svn.freebsd.org address
host svn.freebsd.org | awk '/has address/ { print $4 " svn.freebsd.org"; exit }' >> /etc/hosts
svn checkout --non-interactive --trust-server-cert -r 423390 https://svn.freebsd.org/ports/head /usr/ports
sed -i -e '/svn.freebsd.org/d' /etc/hosts
export BATCH=yes
# compilation of virtualbox-ose fails without bulding python3 port first
cd /usr/ports/lang/python3 && make install clean
cd /usr/ports/emulators/virtualbox-ose && make install clean
pkg -o ASSUME_ALWAYS_YES=YES install phpvirtualbox apache24 mod_php56
pkg -o ASSUME_ALWAYS_YES=YES clean
pw groupadd -n vbox -g 1001
echo "VBox123" | pw useradd -n vbox -c vbox -u 1001 -s /bin/sh -m -g vbox -G vboxusers -d /home/vbox -h -
sed -i -e 's/'\''pass'\'';/'\''VBox123'\'';/' /usr/local/www/phpvirtualbox/config.php
sed -i -e 's/apache24\/data/phpvirtualbox/g' /usr/local/etc/apache24/httpd.conf
sed -i -e '/AddType application\/x-gzip \.gz \.tgz/a\
\
AddType application\/x-httpd-php .php\
AddType application\/x-httpd-php-sources .phps\
' /usr/local/etc/apache24/httpd.conf
sed -i -e 's/#ServerName www.example.com:80/ServerName 127.0.0.1:80/' /usr/local/etc/apache24/httpd.conf
sed -i -e 's/@constant('\''PHPVBOX_VER'\'')/"5.1-0"/' /usr/local/www/phpvirtualbox/endpoints/api.php
sysrc apache24_enable="YES"
sysrc vboxwebsrv_user="vbox"
sysrc vboxwebsrv_enable="YES"
service vboxwebsrv start
service apache24 start
echo Done. Connect to phpvirtualbox at http://`ifconfig -a | grep inet | grep -v inet6 | grep -v 127.0.0.1 | awk '{print $2}'` with user "admin" password "admin"

sh vboxcreate.sh

In FreeNAS web interface
========================
Add Tunable, type "Loader":
vboxdrv_load="YES"
Add Tunable, type "rc.conf":
vboxnet_enable="YES"

and reboot to load vbox drivers.

I haven't tested it fully yet, as I don't have Freenas 11 on physical machine. Only running under other virtualbox and as it does not allow nested virtualisation, I cannot check it if it really works.

Please report back if you try it if it works or you suggest some changes.

Enjoy!

Thank you, Kam, you saved me a lot of hours. I'm using it on freenas 11 and everything works fine. The only problem I'm having so far is that I'm unable to run x64 gusts. I'm always getting this error; "AMD-V is being used by another hypervisor (VERR_SVM_IN_USE). VirtualBox can't enable the AMD-V extension. Please close all other virtualization programs. (VERR_SVM_IN_USE)" I'm getting this error even if there are no bhyve vm up and running. I think I need to completely switch off bhyve but I don't know how to do this. Could anybody help me on that?

Regards, Dmitry

 

[rglover666]

I have not found a way to handle this seamlessly but I just manually unload vmm.ko after every restart of freenas by opening a shell session and issuing the command 'kldunload vmm.ko'. If anyone knows a way to handle this automatically I would be grateful to hear it.

 

[rdmitry0911]

I have not found a way to handle this seamlessly but I just manually unload vmm.ko after every restart of freenas by opening a shell session and issuing the command 'kldunload vmm.ko'. If anyone knows a way to handle this automatically I would be grateful to hear it.

I naven't find it either. The workaround that works for me is to put kldunload vmm in jail-pre-start file and kldload vmm in jail-post-stop file in phpvirtulabox jail config directory. This allows all the x64 VBox machines to start if no freenas vm is running

 

[rglover666]

This appears to be broken again with 11.1 and I don't know how to determine which SVN version to grab to fix it. KAM do you know how to figure this out?

Thanks

 

[amires]

This appears to be broken again with 11.1 and I don't know how to determine which SVN version to grab to fix it. KAM do you know how to figure this out?

Thanks

Try revision 446841 which corresponds to VirtualBox 5.1.26

 

[Kam]

Try revision 446841 which corresponds to VirtualBox 5.1.26

Or revision 449436 which is for VirtualBox 5.1.26_1

 

[Kam]

But there is a bug
Bug #27247 FreeNAS 11-STABLE running FreeBSD 11.1-STABLE r321665+d4625dcee3e installing 11.0-RELASE-p1 userland
which can make problems, as new jail is based on old 11.0 release and not 11.1. It will be better to wait to 11.1-U1 where it should be fixed. I always wait for first or second update of each new release anyway.

 

[amires]

I installed virtualbox-ose-kmod-5.1.26 and virtualbox-ose-nox11-5.1.26 from ports for FreeNAS 11.1 and it is working fine however I have this issue that I cannot shutdown any guest.
Whenever I issue the shutdown command from inside guest os, virtual machine state in web interface changes to stopping and get stuck in that state. I then have to manually kill VBoxHeadless process from shell.

Update 1 : I found the reason. It is vnc related. if vnc is still connected when guest is shutting down, then the VBoxHeadless process hangs. However if I quickly close vnc window after issuing shutdown command then the shutdown process will finish properly.

Update 2 : I think this is the same bug I am experiencing : https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=213019