Pass through a NIC to a VNET jail without using a bridge

[subnetspider]

Hello everyone,

I tried to create an iocage jail on my TrueNAS CORE and pass through one of the hosts physical network cards to it (no vnet:bridge0).

Unfortunately, iocage won't let me do that, saying that VNET interfaces have to be assigned as pairs (with epair) or doing it another way, it fails to start the jail due to "VNET failures" (no bridge).

I know that it is possible to do that with "vanilla" FreeBSD Jails, but don't know if the iocage developers have build in support for that.

Can someone help me shine light on this?
Thanks in advance. :)

 

[Patrick M. Hausen]

That's fundamentally impossible. You can do that with a VM, not with a jail. What's wrong with a bridge? You can use a dedicated interface, the NAS host does not need to have an IP address on that network. It's all just layer 2.

EDIT: it seems that is possible with jails, I doubt iocage can do it, though. More research needed.

 

[subnetspider]

That's fundamentally impossible. You can do that with a VM, not with a jail. What's wrong with a bridge? You can use a dedicated interface, the NAS host does not need to have an IP address on that network. It's all just layer 2.

It is possible, I just did it yesterday on my FreeBSD jail server and it worked fine (with a BastilleBSD jail and an Intel i340-T4 NIC), I could even use SLAAC and iperf3 tests reported full gigabit.

Currently all my jails are on a dedicated FreeBSD server, which hardware is completely overkill, so I want to migrate all my jails to TrueNAS to save power.

A bridge won't do the trick as I want to manage my (nested) jails on TrueNAS CORE with bastille.
They are all VNET jails on different VLANs, and I am not shure how well I can attach multiple VLAN interfaces to a single bridge.

I want to use a iocage jail with nested bastille jails to have as little overhead as possible.

 

[Patrick M. Hausen]

It is possible, I just did it yesterday on my FreeBSD jail server and it worked fine (with a BastilleBSD jail and an Intel i340-T4 NIC), I could even use SLAAC and iperf3 tests reported full gigabit.

How did you do it, then?

 

[subnetspider]

How did you do it, then?

Like this (jail.conf):

Code:

testjail {
  devfs_ruleset = 13;
  enforce_statfs = 2;
  exec.clean;
  exec.consolelog = /var/log/bastille/testjail_console.log;
  exec.start = '/bin/sh /etc/rc';
  exec.stop = '/bin/sh /etc/rc.shutdown';
  host.hostname = testjail;
  mount.devfs;
  mount.fstab = /usr/local/bastille/jails/testjail/fstab;
  path = /usr/local/bastille/jails/testjail/root;
  securelevel = 2;

  vnet;
  vnet.interface = igb3;
}

As soon as I start the VNET jail, the interface igb3 disappears from the host and is attached to the jail, exactly like when using epair, where e0b_interface-name disappears from the host and e0a_interface-name is attached to a bridge.

Output of ifconfig inside the jail:

Code:

root@testjail:~ # ifconfig
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2d
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
igb3: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=4e503bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether 00:19:99:d9:e1:6b
        inet 10.1.20.50 netmask 0xffffff00 broadcast 10.1.20.255
        inet6 fe80::219:99ff:fed9:e16b%igb3 prefixlen 64 scopeid 0x4
        inet6 2003:____:____:____:219:99ff:fed9:e16b prefixlen 64 autoconf pltime 14400 vltime 86400
        inet6 fd5e:d1c2:c9de:20:219:99ff:fed9:e16b prefixlen 64 autoconf pltime 14400 vltime 86400
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

Output if ifconfig on the Host (filtered):

Code:

admin@localhost:~ % ifconfig | grep igb
igb0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
igb1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
igb2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        laggport: igb0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>                                                                    
        laggport: igb1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>

Code:

admin@localhost:~ % ifconfig igb3
ifconfig: interface igb3 does not exist

 

[Patrick M. Hausen]

Interesting - thanks.

 

[subnetspider]

Interesting - thanks.

Agreed, I only found out about this a few months ago when reading "Absolute FreeBSD" and "FreeBSD Mastery: Jails" by MWL.