11.2 - rancher can't get out

[mmeasel]

I'm trying out 11.2 and the interface is delicious, the vm/docker setup is very nice, but the networking is making me want to pull my hair out.
I'm running FN in my cloud (skytap) and I've used it successfully for a few years, mostly providing storage to other vm's.
I followed the instructions for setting up docker/rancher and I can get into the console and ping the 172 address but I CAN'T GET OUT.
I've tried several different interfaces/combinations and the best I've come up with is having two interfaces (vmxnet3). When I have two, I can at least ssh into the container - on the second interface. Doesn't seem to matter what network/subnet (10.x, 192.x, etc) I use.

1) do you need to define interfaces thru the gui ? ( before when I've done it, it just seems to confuse things )
2) is there some firewall thing that I can't find ?
* I see the tap interfaces starting and stopping when I start/stop rancher.
* Traceroute takes 1-3 secs to show that it stopped at the interface.
Any help would be greatly appreciated.

 

[sretalla]

1) do you need to define interfaces thru the gui ? ( before when I've done it, it just seems to confuse things )
2) is there some firewall thing that I can't find ?
* I see the tap interfaces starting and stopping when I start/stop rancher.
* Traceroute takes 1-3 secs to show that it stopped at the interface.

1) not needed.
2) there is one, but again you shouldn't need to look at it.

Look at ifconfig from your FreeNAS host and see if you have bridge0 with a member of your nic (em0 or igb0... maybe vmx0 if you're in the cloud on a VM) and at least one tap interface. if not, you'll need to look at setting that up as a tunable or at least understanding why it isn't happening automatically... you can do something like ifconfig bridge0 addm tap1 up to test adding it if it's missing.

 

[mmeasel]

thanks sretalla, this is what the interface, bridge and tap look like
vmx1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=200099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,RXCSUM_IPV6> ether 00:50:56:26:cd:7f hwaddr 00:50:56:26:cd:7f inet 192.168.0.1 netmask 0xfffffff8 broadcast 192.168.0.7 nd6 options=9<PERFORMNUD,IFDISABLED> media: Ethernet autoselect status: active tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80000<LINKSTATE> ether 00:bd:a8:32:ff:00 hwaddr 00:bd:a8:32:ff:00 nd6 options=1<PERFORMNUD> media: Ethernet autoselect status: active groups: tap Opened by PID 3713 bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 02:a5:e3:fe:80:00 nd6 options=1<PERFORMNUD> groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: vmx1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 2 priority 128 path cost 2000 member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 5 priority 128 path cost 2000000

I gave the bridge an IP address (192.168.0.5) and I can successfully ping it from the guest, but can't ping my gateway at 192.168.0.6 - or anything else for that matter.

Really frustrated.

 

[sretalla]

Do you have VNET set to on? and allow_raw_sockets set to 1?

 

[mmeasel]

Um, those appear to be iocage settings. I'm using 11.2 beta and there's no mention of having to configure iocage or jails or any of that.
Am I missing something ?

 

[sretalla]

Well, duh (to me)... I'm replying to too many threads at the same time and mixing it up... that's the advice I would give for an iocage jail with the same symptoms, not a docker VM like you're working with.

I haven't run into this problem with my rancher VMs, but I do run at least one jail on all of those FreeNAS boxes alongside it, so can't say what it looks like without jails involved.

Hopefully this is the spot where somebody running it like you but with no issue can chime in...