11.2 U2 breaks VNET

[Ixian]

U2 was released earlier today for 11.2 and several of us in the announcement thread have discovered that it breaks VNET. Any jail, etc. using VNET will be unreachable even though they do start.

Just posting as a warning to those who haven't upgraded yet. It's officially available through the GUI but I'd hold off. Some bug slipped through.

I rolled back to U1.

 

[dlavigne]

Once we've pinned down the cause and workaround (which jails/plugins are affected), we'll add the details to the Known Impacts section of the Release Notes.

 

[MoonPie]

Having same issue with VNET. Rolling back to U1. If you need any info I am willing to upgrade and test.

 

[Ixian]

rrdtool segfaults and exits (11) and spams the console log as well with U2.

 

[DoritoSlayer]

I've experienced the same with my Plex plugin/jail after upgrading to 11.2-U2.
Trying to install the Plex plugin again (besides the existing one) creates a plex_2 plugin and jail, but weirdly the original one is gone... The install also failed both that secondary install and one I tried after deleting the plugin and jail entirely. Not sure if it's connected, but thought I'd mention it.

Could anyone explain how I can downgrade back to U1?
Sorry for the newbie question :)

 

[sretalla]

Could anyone explain how I can downgrade back to U1?

It should be as simple as interrupting the boot process and selecting the previous boot environment.

I also note that in addition to breaking the VNET on an existing 11.1-RELEASE jail I had working, I am also unable to make VNET work on a fresh jail with 11.2-RELEASE.

In both cases, it seems I only have the loopback interface (no epair) inside the jail.

Here's the output from the command line creation which produces the jail with ifconfig shown after...

Code:

root@freenas:~ # iocage create -n "newjail" -r 11.2-RELEASE defaultrouter="192.168.1.1" dhcp="on" vnet="on" bpf="yes" allow_raw_sockets="1" boot="on"
newjail successfully created!
* Starting nut
  + Started OK
Traceback (most recent call last):
  File "/usr/local/bin/iocage", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/iocage_cli/create.py", line 164, in cli
    basejail=basejail, thickjail=thickjail, empty=empty)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 611, in create
    clone=clone,
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_create.py", line 87, in create_jail
    return self._create_jail(jail_uuid, location)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_create.py", line 559, in _create_jail
    silent=self.silent)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 66, in __init__
    self.__start_jail__()
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 415, in __start_jail__
    vnet_err = self.start_network(vnet)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 598, in start_network
    err = self.start_network_interface_vnet(nic, net_configs, jid)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 624, in start_network_interface_vnet
    membermtu = self.find_bridge_mtu(bridge)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 975, in find_bridge_mtu
    su.check_call(bridge_cmd, stdout=su.PIPE, stderr=su.PIPE)
UnboundLocalError: local variable 'bridge_cmd' referenced before assignment

ifconfig:

Code:

root@newjail:~ # ifconfig
lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
root@newjail:~ # 

I guess the errors during jail creation are the cause (even though the jail is created and does start)

 

[Alecmascot]

Could anyone explain how I can downgrade back to U1?

Use System =>> Boot Environment

 

[sretalla]

I can confirm that some old warden jails on another of my systems can start with VNET and are working fine, so it's only iocage impacted as far as I can see.

 

[Lothian]

I also get the error "UnboundLocalError: local variable 'bridge_cmd' referenced before assignment" when I try to start my plex plugin. I ended up reverting back to the old version. Things run now. I'll hold off on upgrading now until I see that this is fixed.

 

[pixel8383]

Same to me. Rolled back to 11.2-U1.

Could anyone explain how I can downgrade back to U1?
Sorry for the newbie question :)

Code:

beadm list

will be helpful if you are managing the NAS on ssh.

 

[Zwck]

good to know, i hope a quick fix will be released soon!

 

[Ixian]

Open bug report is here.

May be related to this (fixed) issue with iocage. Which in turn may have been triggered by this update in U2 though that is pure speculation on my part.

Workaround is to disable VNET and assign static ips directly off the host interface for iocage jails. This may not be desirable/possible for some users and may cause other problems, so proceed only if you know what you are doing and understand the risks. Otherwise I'd recommend holding off on installing U2.

 

[shumi]

Open bug report is here.

May be related to this (fixed) issue with iocage. Which in turn may have been triggered by this update in U2 though that is pure speculation on my part.

Workaround is to disable VNET and assign static ips directly off the host interface for iocage jails. This may not be desirable/possible for some users and may cause other problems, so proceed only if you know what you are doing and understand the risks. Otherwise I'd recommend holding off on installing U2.

Thank you lxian.

Assigning IPs directly on the host resolves the following issue ?

Code:

local variable 'bridge_cmd' referenced before assignment

Error: concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/concurrent/futures/process.py", line 175, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/local/lib/python3.6/site-packages/middlewared/worker.py", line 128, in main_worker

and if you may please paste a screenshot on how to assign a static IP on the jail ?

much appreciated!

 

[Yorick]

Crosspost from release thread:

And we have a solution thanks to David Meier. Stop all jails, edit one jail's properties, go to Network Properties, set vnet_default_interface to auto, from none. Save - you may need to uncheck and recheck DHCP Autoconfigure in Basic before Save becomes available. Start your jail(s).

In his testing, making this change on one jail solved it for all.

Edit: This works for jail install as well. Choose Advanced, then set vnet_default_interface to auto and Save when installing a jail.

 

[Ixian]

Appears to work for me as well. Thanks :)

 

[Rafal Lukawiecki]

Crosspost from release thread:

And we have a solution thanks to David Meier. Stop all jails, edit one jail's properties, go to Network Properties, set vnet_default_interface to auto, from none. Save - you may need to uncheck and recheck DHCP Autoconfigure in Basic before Save becomes available. Start your jail(s).

Same issue. Save button not available after changing vnet_default_interface and even after checking/rechecking other boxes. Rolled back to U1 for now.

 

[Yorick]

You could try changing that property in U1, then upgrading to U2.

 

[Ixian]

Change the interface to your physical nic (like igb0, or whatever it is) save it, then edit again. I was able to change and save vnet_default_interface after that.

 

[ByerRA]

Thanks, it worked for me, my jails are back up and running :)

 

[Rafal Lukawiecki]

You could try changing that property in U1, then upgrading to U2.

Just tried, but "auto" option is not available to me in U1. Only ix0, ix1, igb0, igb1, and bridge0. Will try one of those for now. "auto" seems to have been added in U2.