multiple interfaces in jails.

da_da · Apr 7, 2021

I am trying to set up two interfaces on a jail. One uses a nic and the other uses another. The issue I am having is that the one nic with the default gateway works but not the other nic. I am unable to ping other devices in that subnet.

epair0b - 1.1.1.24/24
epair1b - 5.5.5.56/24 DG- 5.5.5.1

I can get to the internet on epair1b, but I cannot ping epair0b from that network.

I am also unable to ping the jails from the nas itself. NAS IP is 1.1.1.33/24

Patrick M. Hausen · Apr 7, 2021

This works perfectly well in FreeBSD but needs a bit more preparation in my experience. TrueNAS tries to do many things automatically to get the simple case of one NIC, a couple of jails, all one network covered as conveniently as possible.

TrueNAS connects jails by the means of the bridge(4) interface. I don't have time for a concise write up for your case at the moment, but might find enough information in this thread, specifically in the posts by me

Ubiquiti Unifi Controller on FreeNAS

Is it possible and/or advisable to run a Unifi controller on a FreeNAS server? I found this video relating to doing this on FreeBSD, for reference: https://www.youtube.com/watch?v=NsIdeqsx4zs

www.truenas.com

Rough outline:

1. Boot without jails enabled
2. Create bridge interfaces for both your physical interfaces manually.
3. If your TrueNAS itself has got IP addresses on the physical interfaces, move them to the bridge interface instead.
4. Reboot, test connectivity for the NAS.
5. Configure the vnet and bridge interfaces for your jail like in the screenshot. Change "auto" for the VNET default interface to "none"!

da_da · Apr 7, 2021

Thank you for the quick response. I believe I have completed all that however...

My interfaces are
vnt0:bridge0,vnet1:bridge1

Is that an issue? do I need to create another bridge?

Patrick M. Hausen · Apr 7, 2021

Nope, if the first is vnet and not vnt, that is well. The bridges are configured statically in Network --＞ Interfaces?

You must create the bridges, not just assign them in the jail's config. And remove "auto" from the default vnet interface.

da_da · Apr 7, 2021

Excellent. Thank you very much... one more quick question...

How do I create a static route for the jail..

as an example:

1.2.3.55/24 is connected on the 1.1.1.24/24 network router. How do I forward traffic to that.

Patrick M. Hausen · Apr 7, 2021

Edit the file /etc/rc.conf inside the jail. Put these lines in there:

Code:

static_routes="anameyoudefine"
route_anameyoudefine="-net 1.2.3.0/24 1.1.1.24"

Stop and restart your jail.

Check routes with netstat -nr inside your jail.

da_da · Apr 7, 2021

Yeah. Thanks I did try the other way using the route add -net was not sure if that's right or stays persistent

da_da · Apr 7, 2021

I am not sure if I should migrate the interface TrueNAS is on to the bridge... not sure if there's a process without getting disconnected.

Patrick M. Hausen · Apr 7, 2021

It's right but not persistent

The rc.conf entry takes care of that route command on every startup of the jail.

Synsir · May 25, 2022

Hi,
I am trying to this this up for some time now and no googling gives me an idea whats the problem is.
I have to Nics, one is supposed to connect to a switch with some security cameras, no gateway, but a different address space.

The Nics are not assigned ip addresses and connected to bridges.
igb0 -> bridge10
igb1 -> bridge20

Currently, the NAS interface and some jails run on bridge10. In VNET mode with their own IP addresses, there is no problem.
Now the zoneminder jail is supposed to see both bridges, but I can't get it to see the bridge20.

I disabled hardware offloading, I added this line in the configuration

vnet0:bridge10,vnet1:bridge20

What can I try?

root@zoneminder:~ # ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether e2:d5:5e:40:ed:c3
hwaddr 02:e8:23:53:a3:0b
inet 192.168.178.225 netmask 0xffffff00 broadcast 192.168.178.255
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=1<PERFORMNUD>

❯ ifconfig
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: igb0 regular home network
options=8120b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
ether 4c:52:62:45:97:47
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=9<PERFORMNUD,IFDISABLED>
igb1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: igb1 security cam network
options=8120b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER>
ether 4c:52:62:45:97:48
media: Ethernet autoselect
status: no carrier
nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff000000
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
groups: pflog
bridge10: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:16:99:00:88:0a
inet 192.168.178.252 netmask 0xffffff00 broadcast 192.168.178.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: vnet0.14 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 12 priority 128 path cost 2000
member: vnet0.8 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 10 priority 128 path cost 2000
member: vnet0.5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 11 priority 128 path cost 2000
member: vnet0.3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 9 priority 128 path cost 2000
member: vnet0.2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 8 priority 128 path cost 2000
member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 7 priority 128 path cost 2000
member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 55
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
bridge20: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:16:99:00:88:14
inet 192.168.1.240 netmask 0xffffff00 broadcast 192.168.1.255
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto stp-rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: igb1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 2 priority 128 path cost 2000000
groups: bridge
nd6 options=9<PERFORMNUD,IFDISABLED>
vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: emby-server-new as nic: epair0b
<redact>
vnet0.2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: handbrake as nic: epair0b
<redact>
vnet0.3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: kimai as nic: epair0b
<redact>
vnet0.5: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: syncthing-jail as nic: epair0b
<redact>
vnet0.8: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: nginx-reverse as nic: epair0b
<redact>
vnet0.14: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: associated with jail: zoneminder as nic: epair0b
options=8<VLAN_MTU>
ether e2:d5:5e:40:ed:c2
hwaddr 02:e8:23:53:a3:0a
groups: epair
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active
nd6 options=1<PERFORMNUD>

I know, currently, igb1's link is down, there is no cable in it since the old server is currently on that switch. But that shouldn't affect the association of the jail to the bridge20, right?

Patrick M. Hausen · May 26, 2022

No, it should not. Please post a full iocage get all for that jail. And please use code tags instead of quote for command output.

Synsir · May 26, 2022

Code:

❯ iocage get all zoneminder
CONFIG_VERSION:28
allow_chflags:0
allow_mlock:0
allow_mount:1
allow_mount_devfs:0
allow_mount_fusefs:0
allow_mount_nullfs:0
allow_mount_procfs:0
allow_mount_tmpfs:1
allow_mount_zfs:0
allow_quotas:0
allow_raw_sockets:1
allow_set_hostname:1
allow_socket_af:0
allow_sysvipc:0
allow_tun:0
allow_vmm:0
assign_localhost:0
available:readonly
basejail:1
boot:1
bpf:1
children_max:0
comment:none
compression:lz4
compressratio:readonly
coredumpsize:off
count:1
cpuset:off
cputime:off
datasize:off
dedup:off
defaultrouter:192.168.178.1
defaultrouter6:auto
depends:none
devfs_ruleset:4
dhcp:0
enforce_statfs:0
exec_clean:1
exec_created:/usr/bin/true
exec_fib:0
exec_jail_user:root
exec_poststart:/usr/bin/true
exec_poststop:/usr/bin/true
exec_prestart:/usr/bin/true
exec_prestop:/usr/bin/true
exec_start:/bin/sh /etc/rc
exec_stop:/bin/sh /etc/rc.shutdown
exec_system_jail_user:0
exec_system_user:root
exec_timeout:60
host_domainname:none
host_hostname:zoneminder
host_hostuuid:zoneminder
host_time:1
hostid:03d502e0-045e-054f-7d06-110700080009
hostid_strict_check:0
interfaces:vnet0:bridge10,vnet1:bridge20
ip4:new
ip4_addr:192.168.178.225/24
ip4_saddrsel:1
ip6:new
ip6_addr:none
ip6_saddrsel:1
ip_hostname:0
jail_zfs:0
jail_zfs_dataset:iocage/jails/zoneminder/data
jail_zfs_mountpoint:none
last_started:2022-05-26 10:15:44
localhost_ip:none
login_flags:-f root
mac_prefix:e2d55e
maxproc:off
memorylocked:off
memoryuse:off
min_dyn_devfs_ruleset:1000
mount_devfs:1
mount_fdescfs:1
mount_linprocfs:0
mount_procfs:0
mountpoint:readonly
msgqqueued:off
msgqsize:off
nat:0
nat_backend:ipfw
nat_forwards:none
nat_interface:none
nat_prefix:172.16
nmsgq:off
notes:none
nsem:off
nsemop:off
nshm:off
nthr:off
openfiles:off
origin:readonly
owner:root
pcpu:off
plugin_name:zoneminder
plugin_repository:https://github.com/ix-plugin-hub/iocage-plugin-index.git
priority:99
pseudoterminals:off
quota:none
readbps:off
readiops:off
release:12.3-RELEASE-p5
reservation:none
resolver:/etc/resolv.conf
rlimits:off
rtsold:0
securelevel:2
shmsize:off
stacksize:off
state:up
stop_timeout:30
swapuse:off
sync_state:none
sync_target:none
sync_tgt_zpool:none
sysvmsg:new
sysvsem:new
sysvshm:new
template:0
type:pluginv2
used:readonly
vmemoryuse:off
vnet:1
vnet0_mac:e2d55e40edc2 e2d55e40edc3
vnet0_mtu:auto
vnet1_mac:none
vnet1_mtu:auto
vnet2_mac:none
vnet2_mtu:auto
vnet3_mac:none
vnet3_mtu:auto
vnet_default_interface:none
vnet_default_mtu:1500
vnet_interfaces:none
wallclock:off
writebps:off
writeiops:off

Patrick M. Hausen · May 26, 2022

Try iocage set "ip4_addr=vnet0|192.168.178.225/24,vnet1|<some other address>/24" zoneminder and restart the jail.

Synsir · May 26, 2022

Ok, thanks! this did the trick to get the interface in the jail. I can also ping the bridge20.
I connected the 2nd physical interface to the switch and added these routes in the rc.conf of the jail.

Code:

# routing secu cameras
#static_routes="secucamnet"
#route_secucamnet="-net 192.168.1.0/24 -iface epair1b"

static_routes="cam1 cam2"
route_cam1="-host 192.168.1.121 -iface epair1b"
route_cam2="-host 192.168.1.122 -iface epair1b"

but it didn't work. Cant ping the cameras. On the switch, I can see some packets send and received with no errors, but thats about all the info I can get there.

Code:

root@zoneminder:/ # netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.178.1      UGS     epair0b
127.0.0.1          link#1             UH          lo0
192.168.1.0/24     link#4             U       epair1b
192.168.1.121      e2:d5:5e:db:a7:27  UHS     epair1b
192.168.1.122      e2:d5:5e:db:a7:27  UHS     epair1b
192.168.1.242      link#4             UHS         lo0
192.168.178.0/24   link#3             U       epair0b
192.168.178.225    link#3             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
::1                               link#1                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#1                        U           lo0
fe80::1%lo0                       link#1                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0
root@zoneminder:/ # ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=0<> metric 0 mtu 33160
        groups: pflog
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether e2:d5:5e:40:ed:c3
        hwaddr 02:e8:23:53:a3:0b
        inet 192.168.178.225 netmask 0xffffff00 broadcast 192.168.178.255
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
epair1b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=8<VLAN_MTU>
        ether e2:d5:5e:db:a7:27
        hwaddr 02:1d:ef:e8:c6:0b
        inet 192.168.1.242 netmask 0xffffff00 broadcast 192.168.1.255
        groups: epair
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        nd6 options=1<PERFORMNUD>
root@zoneminder:/ # ping 192.168.1.240
PING 192.168.1.240 (192.168.1.240): 56 data bytes
64 bytes from 192.168.1.240: icmp_seq=0 ttl=64 time=0.283 ms
64 bytes from 192.168.1.240: icmp_seq=1 ttl=64 time=0.253 ms
^C
--- 192.168.1.240 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.253/0.268/0.283/0.015 ms
root@zoneminder:/ # ping 192.168.1.121
PING 192.168.1.121 (192.168.1.121): 56 data bytes
^C
--- 192.168.1.121 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss

Synsir · May 26, 2022

Ok, I made a mistake configuring the switch.
I made the port I connected to an untagged member of the security camera network VLAN ID20 and thought that was enough.
Put the 802.1Q VLAN PVID Setting for that port was still VLAN ID10. I changed that to 20 and now it works.

Also I reverted the route setting, since "route get 192.168.1.121" indicated that I don't need it.

Thank you very much Patrick!

Important Announcement for the TrueNAS Community.

multiple interfaces in jails.

da_da

Explorer

Patrick M. Hausen

Hall of Famer

Ubiquiti Unifi Controller on FreeNAS

da_da

Explorer

Patrick M. Hausen

Hall of Famer

da_da

Explorer

Patrick M. Hausen

Hall of Famer

da_da

Explorer

da_da

Explorer

Patrick M. Hausen

Hall of Famer

Synsir

Cadet

Patrick M. Hausen

Hall of Famer

Synsir

Cadet

Patrick M. Hausen

Hall of Famer

Synsir

Cadet

Synsir

Cadet

Similar threads