Can't get Jails to work with VLAN

S

sotiris.bos

Explorer
Joined
Jun 12, 2018
Messages
56
Hello, I am going nuts over this.

FreeNAS 11.2

I have a VLAN set as an interface for FreeNAS. I can access the UI from the VLAN IP and performing traceroute from my computer I can see that it is going through the router (instead of directly connecting like on the LAN) so no problems with the network and VLAN situation.

I am trying to set up a jail that will use the VLAN (vlan200) instead of the LAN.

I have created a new bridge called bridge2 and I have added the VLAN interface to the bridge.

I am using VNET.

I have set up an interface for the jail as follows: vnet0:bridge2

I tried using DHCP (DHCP server is running on the VLAN at 10.0.11.1/24) but the jail cannot reach the router at 10.0.11.1 or any other addresses apart from the FreeNAS VLAN IP.

I tried setting a static IP at 10.0.11.251/24 (is outside the DHCP range) and setting the default router to 10.0.11.1. Jail still can only reach 10.0.11.241 which is the IP of the FreeNAS VLAN interface through the bridge. Cannot reach any other addresses, including 10.0.11.1. I tried it with IPV4 Interface of vlan200, vnet0, bce0.

allow raw sockets is on,
vnet_default_interface under Network Properties was set to either none or vlan200, didn't make a difference.

Can't ping or traceroute anything other than 10.0.11.241 (FreeNAS VLAN IP).

Funny thing is I had it working before but I think that the bridge (bridge2) also included bce0 (default LAN interface) which defeats the purpose of the VLAN.

Help


Edit: Here is the tcpdump for dhcp ran on the router:

Code:
listening on igb2.200, link-type EN10MB (Ethernet), capture size 262144 bytes
17:32:04.392005 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:32:05.394185 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300
17:32:10.444763 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:32:10.445331 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300
17:32:19.464675 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:32:19.465494 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300
17:32:34.469420 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:32:34.470015 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300



Here is the same tcpdump for dhcp ran on the freenas box for bridge2 interface:


Code:
listening on bridge2, link-type EN10MB (Ethernet), capture size 262144 bytes
17:32:04.393857 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:32:10.446557 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:32:19.466480 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:32:34.471162 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300


Edit.2: Works fine on LAN with bce0 being a DHCP Interface. Does not work on VLAN (vlan200) because it has a statically assigned IP. Does the "DHCP"option on the UI under "Interfaces" actually mean that you can only use one interface for DHCP requests for jails as well, not only for the FreeNAS box itself?






Edit.3:
After setting the bce0 (LAN) interface as static and vlan200 interface as DHCP, here are the tcpdumps for DHCP on both the router and the FreeNAS box. Tried it multiple times, seems like the first DHCP offer makes it through to the bridge2 interface while the next ones don't.

Router:
Code:
listening on igb2.200, link-type EN10MB (Ethernet), capture size 262144 bytes
17:51:50.126951 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:51:50.127465 IP 10.0.11.1.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 300
17:51:50.127795 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:51:50.128179 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300
17:51:56.237995 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:51:56.238565 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300
17:52:03.305127 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:52:03.305745 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300
17:52:10.328294 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:52:10.328977 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300
17:52:21.343562 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:52:21.344157 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300
17:52:40.380702 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:52:40.381386 IP 10.0.11.1.67 > 10.0.11.39.68: BOOTP/DHCP, Reply, length 300


FreeNAS:
Code:
listening on bridge2, link-type EN10MB (Ethernet), capture size 262144 bytes
17:51:50.129170 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:51:50.129971 IP 10.0.11.1.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 300
17:51:50.130115 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:51:56.240232 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:52:03.307409 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:52:10.330581 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:52:21.345833 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300
17:52:40.382944 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 02:ff:60:ae:1b:76, length 300


Edit.4: DHCP option seems irrelevant. I have the same behavior as Edit.3 even with vlan200 Interface as static with no DHCP option ticked after a reboot.
 
Last edited:
M

millst

Contributor
Joined
Feb 2, 2015
Messages
141
The bridge needs to connect the jail interface to a real interface. I would think you'd need to create a VLAN interface through the GUI (Network section). Then, add that VLAN interface to the bridge.
 
S

sotiris.bos

Explorer
Joined
Jun 12, 2018
Messages
56
millst said:
The bridge needs to connect the jail interface to a real interface. I would think you'd need to create a VLAN interface through the GUI (Network section). Then, add that VLAN interface to the bridge.
Click to expand...
I have. I have written so on the fifth paragraph
 
M

millst

Contributor
Joined
Feb 2, 2015
Messages
141
Maybe post your jail network settings and ifconfig output for static IP.
 
HolyK

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
654
Heya,

I might be absolutely OT as i am still on 9 release but take a look on this post (for 9.10 version) and this for 11.1 .... maybe it will help you to move on.

Please share your solution if you make it working as desired :]
 
S

sotiris.bos

Explorer
Joined
Jun 12, 2018
Messages
56
millst said:
Maybe post your jail network settings and ifconfig output for static IP.
Click to expand...

ifconfig run on FreeNAS
Code:
ifconfig
bce0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c00b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
    ether 00:24:e8:2f:37:ee
    hwaddr 00:24:e8:2f:37:ee
    inet 10.0.10.84 netmask 0xffffff00 broadcast 10.0.10.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
bce1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
    ether 00:24:e8:2f:37:f0
    hwaddr 00:24:e8:2f:37:f0
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect
bce2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
    ether 00:24:e8:2f:37:f2
    hwaddr 00:24:e8:2f:37:f2
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect
bce3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
    ether 00:24:e8:2f:37:f4
    hwaddr 00:24:e8:2f:37:f4
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
mlxen0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    ether 00:02:c9:a1:9e:d0
    hwaddr 00:02:c9:a1:9e:d0
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect
    status: no carrier
vlan200: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80001<RXCSUM,LINKSTATE>
    ether 00:24:e8:2f:37:ee
    inet 10.0.11.241 netmask 0xffffff00 broadcast 10.0.11.255
    nd6 options=9<PERFORMNUD,IFDISABLED>
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    vlan: 200 vlanpcp: 0 parent interface: bce0
    groups: vlan
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:0f:8b:28:b7:00
    nd6 options=1<PERFORMNUD>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0:9 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 17 priority 128 path cost 2000
    member: vnet0:8 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 16 priority 128 path cost 2000
    member: vnet0:7 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 15 priority 128 path cost 2000
    member: vnet0:6 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 14 priority 128 path cost 2000
    member: vnet0:5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 13 priority 128 path cost 2000
    member: vnet0:4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 12 priority 128 path cost 2000
    member: vnet0:3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 11 priority 128 path cost 2000
    member: vnet0:2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 10 priority 128 path cost 2000
    member: vnet0:1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 9 priority 128 path cost 2000
    member: bce0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 20000
vnet0:1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: sonarr_iocg
    options=8<VLAN_MTU>
    ether 02:ff:60:04:aa:cf
    hwaddr 02:76:10:00:09:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: tautulli
    options=8<VLAN_MTU>
    ether 02:ff:60:46:b0:16
    hwaddr 02:76:10:00:0a:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: organizrv2
    options=8<VLAN_MTU>
    ether 02:ff:60:b1:d1:6b
    hwaddr 02:76:10:00:0b:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: plex_iocg
    options=8<VLAN_MTU>
    ether 02:ff:60:8f:b2:f5
    hwaddr 02:76:10:00:0c:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:5: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: nginx_iocg
    options=8<VLAN_MTU>
    ether 02:ff:60:4e:bd:e0
    hwaddr 02:76:10:00:0d:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:6: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: transmission
    options=8<VLAN_MTU>
    ether 02:ff:60:03:aa:46
    hwaddr 02:76:10:00:0e:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:7: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: lidarr
    options=8<VLAN_MTU>
    ether 02:ff:60:bf:cd:2e
    hwaddr 02:76:10:00:0f:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:8: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: radarr_iocg
    options=8<VLAN_MTU>
    ether 02:ff:60:5e:f9:c1
    hwaddr 02:76:10:00:10:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
vnet0:9: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: sabnzbd
    options=8<VLAN_MTU>
    ether 02:ff:60:ef:4d:38
    hwaddr 02:76:10:00:11:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
bridge2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:0f:8b:28:b7:02
    nd6 options=1<PERFORMNUD>
    groups: bridge
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: vnet0:10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 19 priority 128 path cost 2000
    member: vlan200 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 7 priority 128 path cost 20000
vnet0:10: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    description: associated with jail: test
    options=8<VLAN_MTU>
    ether 02:ff:60:ae:1b:75
    hwaddr 02:76:10:00:13:0a
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair
epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=8<VLAN_MTU>
    ether 02:ff:60:ae:1b:76
    hwaddr 02:76:10:00:14:0b
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active



ifconfig run in test jail:
Code:
ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
    inet 127.0.0.1 netmask 0xff000000
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    groups: lo
epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=8<VLAN_MTU>
    ether 02:ff:60:ae:1b:76
    hwaddr 02:76:10:00:16:0b
    inet 10.0.11.242 netmask 0xffffff00 broadcast 10.0.11.255
    nd6 options=1<PERFORMNUD>
    media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
    status: active
    groups: epair


ping router from test jail:
Code:
root@test:/ # ping 10.0.11.1
PING 10.0.11.1 (10.0.11.1): 56 data bytes
ping: sendto: Host is down
ping: sendto: Host is down


ping vlan200 FreeNAS IP from test jail:
Code:
root@test:/ # ping 10.0.11.241
PING 10.0.11.241 (10.0.11.241): 56 data bytes
64 bytes from 10.0.11.241: icmp_seq=0 ttl=64 time=0.095 ms
64 bytes from 10.0.11.241: icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from 10.0.11.241: icmp_seq=2 ttl=64 time=0.041 ms
 

Attachments

  • jail1.jpeg
    jail1.jpeg
    51.4 KB · Views: 666
  • jail.jpeg
    jail.jpeg
    46.2 KB · Views: 714
S

sotiris.bos

Explorer
Joined
Jun 12, 2018
Messages
56
HolyK said:
Heya,

I might be absolutely OT as i am still on 9 release but take a look on this post (for 9.10 version) and this for 11.1 .... maybe it will help you to move on.

Please share your solution if you make it working as desired :]
Click to expand...
Tried the 11.1 solution, no luck unfortunately.
 
R

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
I am experiencing the same problem. I have a managed Cisco switch and have setup a trunk on the switch to my Freenas server. I then created a new Vlan interface on Freenas and bridged the Vlan interface with a new bridge that I created. After creating a new jail and attaching the jail to the new bridge I have the same result as @sotiris.bos. I can only ping the Freenas host IP, nothing else.
 
HolyK

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
654
@sotiris.bos Without having access to 11.2 box it is hard to test but i would say something is wrong with the output of your host ifconfig. You have epair0b on both sides (host should have epair0a, jail will have epair0b). Stop the jail and remove the network configuration. Destroy whatever was not removed manually (epair0b, bridge2, vnet0:10). Also if you're not using the VLAN200 for other things i suggest to destroy that as well and re-create it.

Anyway i assume there is something wrong with the way how jails have their interfaces created. Sadly there is no jail.conf in FreeNAS so we can't check (anybody knows where the definitions done via UI are stored if not in jail.conf ?).

I am still too scared to upgrade after reading the "Data lost" threads. More over i am on encrypted pools. But once i upgrade i will need to figure this as well as i am currently running everything over VLANs with various isolation.
 
S

sotiris.bos

Explorer
Joined
Jun 12, 2018
Messages
56
HolyK said:
@sotiris.bos Without having access to 11.2 box it is hard to test but i would say something is wrong with the output of your host ifconfig. You have epair0b on both sides (host should have epair0a, jail will have epair0b). Stop the jail and remove the network configuration. Destroy whatever was not removed manually (epair0b, bridge2, vnet0:10). Also if you're not using the VLAN200 for other things i suggest to destroy that as well and re-create it.

Anyway i assume there is something wrong with the way how jails have their interfaces created. Sadly there is no jail.conf in FreeNAS so we can't check (anybody knows where the defini
Click to expand...
Thank you for your input!

I believe the epair0b interface on the host is related to this:

https://github.com/iocage/iocage/issues/757

because I tried to start the jail with DHCP as well as a different IPV4 Interface and static IP setting prior to posting the data above.

I'll try to recreate the vlan interface and play around with jail settings but it seems to me it is a host problem not routing traffic correctly between VLANs and jails. As I mentioned above on the tcpdumps, some DHCP traffic made it through and some did not when I was testing it.
 
HolyK

HolyK

Ninja Turtle
Moderator
Joined
May 26, 2011
Messages
654
sotiris.bos said:
Thank you for your input!I'll try to recreate the vlan interface and play around with jail settings but it seems to me it is a host problem not routing traffic correctly between VLANs and jails. As I mentioned above on the tcpdumps, some DHCP traffic made it through and some did not when I was testing it.
Click to expand...

Yes the sniffs looks OK. I had issues with VLANs as well. This area is not yet polished. Clean and try again + give us output of ifconfig -a from both host and jail as well as the netstat -nr. Also do the .meta dir still exist? Check if you have ".<jail_name>.meta" directory under jail dataset. If yes could you ZIP it and attach it here? )

Another way would be to remove everything from GUI and try to setup it manually from CLI (jail creation included). I know these changes will not survive reboot but at least to check if it works (so the issue is between GUI and backend) or if it does not work either (so something broken in OS). If it will work then collect data, restart host system, do the setup via GUI again and compare the differences.
 
T

thulle

Cadet
Joined
Jan 23, 2019
Messages
7
I seem to have the same issue. I also found this thread: https://forums.freenas.org/index.php?threads/vlan-issue.60597/ which make it seem like this issue has been around for at least a year.

I have igb0 as an vlan-untagged interface and vlan253 as a vlan-alias with igb0 as parent:

Code:
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6>
        ether 00:25:90:32:ca:4c
        hwaddr 00:25:90:32:ca:4c
        inet 172.17.8.50 netmask 0xffffff00 broadcast 172.17.8.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

vlan253: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=200001<RXCSUM,RXCSUM_IPV6>
        ether 00:25:90:32:ca:4c
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        vlan: 253 vlanpcp: 0 parent interface: igb0
        groups: vlan


The vlan253-interface is then used as a bridge for a VM and shows up as bridge1 with vlan253 and the VM-tap interface as members:

Code:
bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:37:99:93:61:01
        nd6 options=1<PERFORMNUD>
        groups: bridge
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 10 priority 128 path cost 2000000
        member: vlan253 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 9 priority 128 path cost 20000

tap3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 00:bd:15:9a:19:03
        hwaddr 00:bd:15:9a:19:03
        nd6 options=1<PERFORMNUD>
        media: Ethernet autoselect
        status: active
        groups: tap
        Opened by PID 51805




Ie. traffic goes from the vm as follows:
VM -> TAP3 -> bridge1 -> vlan253 -> igb0 with vlan-tag -> DHCP-server
And should go the reverse path back to the vm.

If I tcpdump tap3 I can see the DHCP-request leaving:
Code:
14:53:49.302737 00:a0:98:56:19:5b > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 331: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:a0:98:56:19:5b, length 289


tcpdump on bridge1 sees it pass through:
Code:
14:53:49.302759 00:a0:98:56:19:5b > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 331: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:a0:98:56:19:5b, length 289


tcpdump on vlan253 sees it pass through:
Code:
14:53:49.302751 00:a0:98:56:19:5b > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 331: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:a0:98:56:19:5b, length 289


tcpdump on igb0 sees it pass through with correct vlan-tag, and here the reply arrives with correct vlan-tag.
Code:
14:53:49.302755 00:a0:98:56:19:5b > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 335: vlan 253, p 0, ethertype IPv4, 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:a0:98:56:19:5b, length 289
14:53:49.327676 fc:ec:da:04:b3:b2 > 00:a0:98:56:19:5b, ethertype 802.1Q (0x8100), length 346: vlan 253, p 0, ethertype IPv4, 172.17.253.1.67 > 172.17.253.2.68: BOOTP/DHCP, Reply, length 300


So it seems like the DHCP-reply does make it back to the server, but not back to the vlan253-interface.

The odd thing is that there's other traffic on the vlan that passes through. A multicast packet comes at igb0:
Code:
14:53:43.889485 fc:ec:da:04:b3:b2 > 01:00:5e:00:00:fb, ethertype 802.1Q (0x8100), length 255: vlan 253, p 0, ethertype IPv4, 172.17.253.1.5353 > 224.0.0.251.5353: 0*- [0q] 5/0/0 PTR xxxxxxxxxxxxxxxx-0._spotify-connect._tcp.local., A 172.xx.xx.xx, TXT "CPath=/zc/0" "VERSION=1.0" "Stack=SP", SRV xxxxxxxxxxxxxxxx.local.:46387 0 0, PTR _spotify-connect._tcp.local. (209)


Passes through vlan253:
Code:
14:53:43.889498 fc:ec:da:04:b3:b2 > 01:00:5e:00:00:fb, ethertype IPv4 (0x0800), length 251: 172.17.253.1.5353 > 224.0.0.251.5353: 0*- [0q] 5/0/0 PTR xxxxxxxxxxxxxxxx-0._spotify-connect._tcp.local., A 172.xx.xx.xx, TXT "CPath=/zc/0" "VERSION=1.0" "Stack=SP", SRV xxxxxxxxxxxxxxxx-0.local.:46387 0 0, PTR _spotify-connect._tcp.local. (209)


Goes onto the bridge:
Code:
14:53:43.889509 fc:ec:da:04:b3:b2 > 01:00:5e:00:00:fb, ethertype IPv4 (0x0800), length 251: 172.17.253.1.5353 > 224.0.0.251.5353: 0*- [0q] 5/0/0 PTR xxxxxxxxxxxxxxxx-0._spotify-connect._tcp.local., A 172.xx.xx.xx, TXT "CPath=/zc/0" "VERSION=1.0" "Stack=SP", SRV xxxxxxxxxxxxxxxx-0.local.:46387 0 0, PTR _spotify-connect._tcp.local. (209)


And reaches tap3:
Code:
14:53:43.889556 fc:ec:da:04:b3:b2 > 01:00:5e:00:00:fb, ethertype IPv4 (0x0800), length 251: 172.17.253.1.5353 > 224.0.0.251.5353: 0*- [0q] 5/0/0 PTR xxxxxxxxxxxxxxxx-0._spotify-connect._tcp.local., A 172.xx.xx.xx, TXT "CPath=/zc/0" "VERSION=1.0" "Stack=SP", SRV xxxxxxxxxxxxxxxx-0.local.:46387 0 0, PTR _spotify-connect._tcp.local. (209)


I can also see the traffic just fine in the VM.


So it appears it's just the DHCP-replies that gets lost.
I'm not that familiar with BSD, but I interpret this as network filtering with pf not being active:
Code:
# pfctl -s all
pfctl: /dev/pf: No such file or directory.


Are there any other tunable filters other than pf or should I file this as a bugreport?
 
T

thulle

Cadet
Joined
Jan 23, 2019
Messages
7
Filed as bug #72115. I found a thread on FreeBSD forums that hints that this might be an Intel NIC driver issue.
That doesn't seem the case for @sotiris.bos though, so excuse me for maybe hijacking the thread with something unrelated.
 
Last edited by a moderator:
T

thulle

Cadet
Joined
Jan 23, 2019
Messages
7
As updated in the bug report, switching the vlan-bridge to use any other NIC seems to make this work for me.
Using the primary NIC that FreeNAS uses doesn't work.

@sotiris.bos would you be able to try using a separate port for the VLAN to see if this affects your situation in any way?
 
R

raidflex

Guru
Joined
Mar 14, 2012
Messages
531
thulle said:
As updated in the bug report, switching the vlan-bridge to use any other NIC seems to make this work for me.
Using the primary NIC that FreeNAS uses doesn't work.
Click to expand...

So as of now if you are using a trunk port with a Vlan interface created on Freenas and then bridge that Vlan interface with the jail's vnet interface this does not work when using the primary NIC? Because this is what I experienced, I get no network connectivity in this setup.
 
T

thulle

Cadet
Joined
Jan 23, 2019
Messages
7
raidflex said:
So as of now if you are using a trunk port with a Vlan interface created on Freenas and then bridge that Vlan interface with the jail's vnet interface this does not work when using the primary NIC? Because this is what I experienced, I get no network connectivity in this setup.
Click to expand...

At least using the igb driver with at least a subset of intel cards. @sotiris.bos is using the bce-driver and seem to be having the same issue, but other than that I have no indication it's a generic issue.
Could you post your ifconfig output and try using a secondary port for the vlan?
 
slovdahl

slovdahl

Cadet
Joined
Jun 20, 2016
Messages
1
I think I'm experiencing the same problem, currently running 11.2-U3. Looks like https://redmine.ixsystems.com/issues/72115 was closed a few weeks ago, but it's a bit unclear to me if it actually was fixed, and if yes, in which version? Looks like it isn't possible to comment on the issue after it has been closed.
 
pro lamer

pro lamer

Guru
Joined
Feb 16, 2018
Messages
626
And (every)one needs to click "watch this ticket" again.

Sent from my phone
 
You must log in or register to reply here.

Similar threads

hexley
SOLVED Losing packets between LAGG and VLAN
Replies
3
Views
2K
sean.nutech
S
K
  • Locked
fibs are using wrong arps on replies to incoming connections
Replies
2
Views
1K
kerbstrike
K
A
  • Locked
SOLVED Tagged traffic not copied to a bridge interface
Replies
1
Views
1K
asimov-solensan
A
W
Unable to get past bridge when using vlan in iocage jail
Replies
14
Views
3K
WilbertNL
W
ezra
BOUNTY!! Help wanted, yet another VLAN/Jail topic, tried everything
2
Replies
21
Views
7K
victort
victort
Top