11.2 U2 breaks VNET

[shumi]

Crosspost from release thread:

And we have a solution thanks to David Meier. Stop all jails, edit one jail's properties, go to Network Properties, set vnet_default_interface to auto, from none. Save - you may need to uncheck and recheck DHCP Autoconfigure in Basic before Save becomes available. Start your jail(s).

In his testing, making this change on one jail solved it for all.

Edit: This works for jail install as well. Choose Advanced, then set vnet_default_interface to auto and Save when installing a jail.

Thank you Yorick.

Jails are back up and running. I didn't have the "auto" option, but did change to once of the NIC options.
Can you explain what is the difference if I chose em0 or bridge0 ?

thanks again,

 

[Rafal Lukawiecki]

Just tried, but "auto" option is not available to me in U1. Only ix0, ix1, igb0, igb1, and bridge0. Will try one of those for now. "auto" seems to have been added in U2.

Trying with ix0 failed with error "Llist index out of range" when trying to start the jail. However, after that "auto" has appeared in the list, and I was able to select it, and save, using the checkbox tick/untick method. Jails working again, thank you.

 

[Ixian]

You would want to use your actual adapter of course, assuming it is ix0 (which is for Intel 10Gbe) instead of trying random ones. Though auto should do that for you. Glad to hear it works for you now.

 

[SW77]

I don't have the auto option to select under vnet_default_interface.

 

[Yorick]

@shumi: Like @Ixian says, use your actual physical adapter in that case. em0 is an Intel Gbit interface. In my case physical is lagg0, which contains igb0 and igb1. The names correspond to the driver used for the physical adapter.

 

[indivision]

Same issue here. Is there an estimated schedule for a hotfix? Or, should I roll back and wait?

 

[sretalla]

I confirm the suggested process worked for me also on a jail created via the CLI in my earlier post ... important that the jail is stopped when editing it in the GUI it seems.

 

[biltong]

PSA: There's a "fix" on the bug tracker (https://redmine.ixsystems.com/issues/75648#note-37):

Code:

/bin/sh -c 'export JAILS="$(iocage list -h | cut -f 2)"; for jail in $JAILS; do iocage set vnet_default_interface=auto $jail; done; iocage set vnet_default_interface=auto default'

Worked for me. I ran this and restarted my jails and they are talking again.

 

[Cytomax]

This is a noob question but do you run this on the freenas terminal or in each individual jail

 

[Yorick]

In the FreeNAS terminal. What this does is cycle through each jail, set vnet_default_interface to auto, then change the iocage config so all jails created thereafter will also default to that setting.

 

[pixel8383]

Open bug report is here.

May be related to this (fixed) issue with iocage. Which in turn may have been triggered by this update in U2 though that is pure speculation on my part.

Workaround is to disable VNET and assign static ips directly off the host interface for iocage jails. This may not be desirable/possible for some users and may cause other problems, so proceed only if you know what you are doing and understand the risks. Otherwise I'd recommend holding off on installing U2.

I do confirm using static ips is working for me.

 

[Ixian]

I do confirm using static ips is working for me.

That was just a temp. fix I did to rule out other issues.

You really should do the workaround posted above (configuring auto to default for the vnet interface). In most cases, VNET is preferable for jails. VNET provides jails a full networking stack, which is important for any jail that needs network broadcast or discovery. Plex, the UniFi Controller software, any application that relies on DNLA, jails that use the same common ports, etc.

 

[StephanWolf]

Crosspost from release thread:

And we have a solution thanks to David Meier. Stop all jails, edit one jail's properties, go to Network Properties, set vnet_default_interface to auto, from none. Save - you may need to uncheck and recheck DHCP Autoconfigure in Basic before Save becomes available. Start your jail(s).

In his testing, making this change on one jail solved it for all.

Edit: This works for jail install as well. Choose Advanced, then set vnet_default_interface to auto and Save when installing a jail.

Worked for me.

 

[pixel8383]

That was just a temp. fix I did to rule out other issues.

You really should do the workaround posted above (configuring auto to default for the vnet interface). In most cases, VNET is preferable for jails. VNET provides jails a full networking stack, which is important for any jail that needs network broadcast or discovery. Plex, the UniFi Controller software, any application that relies on DNLA, jails that use the same common ports, etc.

I fixed it now as per your suggestion.
Thank you.

 

[Dave Genton]

It should be as simple as interrupting the boot process and selecting the previous boot environment.

I also note that in addition to breaking the VNET on an existing 11.1-RELEASE jail I had working, I am also unable to make VNET work on a fresh jail with 11.2-RELEASE.

In both cases, it seems I only have the loopback interface (no epair) inside the jail.

Here's the output from the command line creation which produces the jail with ifconfig shown after...

Code:

root@freenas:~ # iocage create -n "newjail" -r 11.2-RELEASE defaultrouter="192.168.1.1" dhcp="on" vnet="on" bpf="yes" allow_raw_sockets="1" boot="on"
newjail successfully created!
* Starting nut
  + Started OK
Traceback (most recent call last):
  File "/usr/local/bin/iocage", line 10, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/iocage_cli/create.py", line 164, in cli
    basejail=basejail, thickjail=thickjail, empty=empty)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/iocage.py", line 611, in create
    clone=clone,
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_create.py", line 87, in create_jail
    return self._create_jail(jail_uuid, location)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_create.py", line 559, in _create_jail
    silent=self.silent)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 66, in __init__
    self.__start_jail__()
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 415, in __start_jail__
    vnet_err = self.start_network(vnet)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 598, in start_network
    err = self.start_network_interface_vnet(nic, net_configs, jid)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 624, in start_network_interface_vnet
    membermtu = self.find_bridge_mtu(bridge)
  File "/usr/local/lib/python3.6/site-packages/iocage_lib/ioc_start.py", line 975, in find_bridge_mtu
    su.check_call(bridge_cmd, stdout=su.PIPE, stderr=su.PIPE)
UnboundLocalError: local variable 'bridge_cmd' referenced before assignment

ifconfig:

Code:

root@newjail:~ # ifconfig
lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
groups: lo
root@newjail:~ # 

I guess the errors during jail creation are the cause (even though the jail is created and does start)

Oddly enough I was having this issue prior to the update, therefore was using a VM vs a jail to host that service/application. Once I updated to U2 the vnet started working for me finally and the jail is online. I did also have to revert back to prior boot environment because SMB service will not start on my machine with U2, but the jail still works since its running update p9 still.

 

[NamoMitK]

rrdtool segfaults and exits (11) and spams the console log as well with U2.

Was trying to diagnose this one on my own thinking it was just me. Thought it might be some corrupted files.

 

[Rilo Ravestein]

I was having similar issues, and the fix was quite easy after all. https://forums.freenas.org/threads/iocage-vnet0-and-bridge0.59964/post-514190

 

[mloiterman]

Should vnet_default_interface be set back to none when this is eventually fixed or is it acceptable to leave it on auto?

 

[RSVP]

Feels like the last year I have been tinkering with jails. Since the change in freebsd. I Rolled back to previous version. Jails stopped working later. Not sure what that is about. Looking forward to getting a stable media centre.

 

[RSVP]

Activated the boot environment back to U2. changed the interface to auto. no go. So in summary cant roll back it seems, and cant work with current update. Was allready struggling with iocage connectivity issues.