iocage jails in VLANs

A

acp

Explorer
Joined
Mar 25, 2013
Messages
71
Reading from your post it similar to what I faced here. I tested it with a VM (XCP-NG 7.6) of 11.2-U2.1. The only way I could get the jail to pull an IP address using DHCP was put an bogus IP on the vlan interface. Once I did that, the jail would have access to the vlan and pull successful. During my testing I noticed that the jail would ask for an IP (as seen on the MikroTik router) but the request would always time out and I would get an error. I also noticed during my testing if I used allowed the switch tag the traffic direct to a given interface it also worked as well. To allow multiple vlan interfaces I need separate bridges for each vlan, but that can be done in the gui interface. It defaults to bridge0, but that field is editable.

I can provide more details if you like.
 
B

b1ackb3ar

Cadet
Joined
Mar 18, 2019
Messages
6
I've also been struggling with this. I tried implementing what I think you guys are doing as a workaround until this is resolved but having no luck. I must be missing something...

- FreeNAS-11.2-U2.1
- iocage Version 1.0 ALPHA 1
- VMWare ESXi 6.7
- vmx0 = VLAN tagged at VMWare vSwitch
- vmx1 = untagged trunk

I can ping successfully to/from the FreeNAS VLAN interface (static or DHCP) to the network router (or any other network endpoint) but cannot ping the jail or ping anything outside the jail from within the jail (setting up a static IP instead of DHCP).

Here's my ifconfig:
Code:
vmx0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=60039b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:0c:29:cf:7e:6c
        hwaddr 00:0c:29:cf:7e:6c
        inet 10.0.10.12 netmask 0xffffff00 broadcast 10.0.10.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: active
vmx1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=60039b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:0c:29:cf:7e:76
        hwaddr 00:0c:29:cf:7e:76
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
vlan900: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=200001<RXCSUM,RXCSUM_IPV6>
        ether 00:0c:29:cf:7e:76
        inet 10.0.90.99 netmask 0xffffff00 broadcast 10.0.90.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: active
        vlan: 900 vlanpcp: 0 parent interface: vmx1
        groups: vlan
bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:1e:01:a8:bd:01
        nd6 options=1<PERFORMNUD>
        groups: bridge
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: vnet0:1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: vlan900 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
vnet0:1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: nextcloud
        options=8<VLAN_MTU>
        ether 02:ff:60:ba:b5:81
        hwaddr 02:67:10:00:06:0a
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        groups: epair


Any help would be greatly appreciated because this is sooo frustrating!

Thanks,
Ryan
 
A

acp

Explorer
Joined
Mar 25, 2013
Messages
71
b1ackb3ar said:
I've also been struggling with this. I tried implementing what I think you guys are doing as a workaround until this is resolved but having no luck. I must be missing something...

- FreeNAS-11.2-U2.1
- iocage Version 1.0 ALPHA 1
- VMWare ESXi 6.7
- vmx0 = VLAN tagged at VMWare vSwitch
- vmx1 = untagged trunk

I can ping successfully to/from the FreeNAS VLAN interface (static or DHCP) to the network router (or any other network endpoint) but cannot ping the jail or ping anything outside the jail from within the jail (setting up a static IP instead of DHCP).

Here's my ifconfig:
Code:
vmx0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=60039b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:0c:29:cf:7e:6c
        hwaddr 00:0c:29:cf:7e:6c
        inet 10.0.10.12 netmask 0xffffff00 broadcast 10.0.10.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: active
vmx1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=60039b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:0c:29:cf:7e:76
        hwaddr 00:0c:29:cf:7e:76
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo
vlan900: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=200001<RXCSUM,RXCSUM_IPV6>
        ether 00:0c:29:cf:7e:76
        inet 10.0.90.99 netmask 0xffffff00 broadcast 10.0.90.255
        nd6 options=9<PERFORMNUD,IFDISABLED>
        media: Ethernet autoselect
        status: active
        vlan: 900 vlanpcp: 0 parent interface: vmx1
        groups: vlan
bridge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:1e:01:a8:bd:01
        nd6 options=1<PERFORMNUD>
        groups: bridge
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: vnet0:1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 6 priority 128 path cost 2000
        member: vlan900 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 4 priority 128 path cost 2000
vnet0:1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: associated with jail: nextcloud
        options=8<VLAN_MTU>
        ether 02:ff:60:ba:b5:81
        hwaddr 02:67:10:00:06:0a
        nd6 options=1<PERFORMNUD>
        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
        status: active
        groups: epair


Any help would be greatly appreciated because this is sooo frustrating!

Thanks,
Ryan
Click to expand...
Are you using a static in the jail or DHCP?
 
B

b1ackb3ar

Cadet
Joined
Mar 18, 2019
Messages
6
acp said:
Are you using a static in the jail or DHCP?
Click to expand...
Tried both. DCHP wouldn't work at all. Just tried static and I'm able to ping between the FreeNAS host VLAN IP and the jail but cannot ping between the jail and any other network asset (including the gateway).

Quick Summary:
- FreeNAS host VLAN <=> jail (Successful ping)
- FreeNAS host VLAN <=> rest of network (Successful ping)
- jail <=> rest of network (cannot ping)

I don't have any static routes specified.
 
A

acp

Explorer
Joined
Mar 25, 2013
Messages
71
b1ackb3ar said:
Got it working! The issue was in VMWare ESXi. Promiscuous Mode needed to be enabled on the vSwitch.

https://www.ixsystems.com/community...network-or-jail-notworking.68626/#post-470664

I was also able to remove the IP config from the FreeNAS VLAN interface. The jail came right up with DHCP.
Click to expand...
I will have to check to see if there is a similar requirement for xcp-ng.

I was planning to put this on my prod machine since I figure out how to make it work in a virtual environment. My prod machine is running on metal so maybe I won't need the IP on the vlan interfaces.
 
A

acp

Explorer
Joined
Mar 25, 2013
Messages
71
So I figured it out. You must specify up in the options on the vlan interfaces and the jail can't be autostarted.
 
ezra

ezra

Contributor
Joined
Jan 15, 2015
Messages
124
I guess it will, just takes a little longer. Had this problem to, if you still have trouble with it i created an auto start snippit.

Code:
#!/bin/bash
declare -a vars
eval "vars=(`/usr/local/bin/iocage list | awk '{ print $4 }' | sed '2d' | grep .`)"
for ((I = 0; I < ${#vars[@]}; ++I )); do
    /usr/local/bin/iocage start "${vars[$I]}"
done
 
  • Like
Reactions: acp
A

acp

Explorer
Joined
Mar 25, 2013
Messages
71
I wrote a script that I put in the task as a startup script. pretty simple. Sleep 60 and start each one manually. I was going to submit a bug report after 11.2-U3 came out and I tested it. Your script it nicer than mine and it will start all of them :)

Thanks for the script.
 
You must log in or register to reply here.

Similar threads

T
VLANs and Jails
Replies
2
Views
989
alexr
alexr
Newfoundland.Republic
VLANs need to have IP addresses?
Replies
7
Views
2K
Newfoundland.Republic
Newfoundland.Republic
M
Putting jails on separate VLANs
Replies
3
Views
2K
medicineman25
M
Kennyvb8
VLAN - explain please on how to do it.
2 3
Replies
45
Views
23K
StarkJohan
StarkJohan
beaster
VLANs Bridges and LAG Interface best practice questions
2
Replies
31
Views
11K
beaster
beaster
Top