Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

Internet loss in jail behind VPN

Sty

Newbie
Joined
Jun 2, 2015
Messages
16
Good evening,

I am unsure what is going on and unfortunately can't find logs that help me out. I am running a FreeNAS with a couple of plugins, Plex and Transmission. I have a pfSense that runs the whole of the network, and one of the things that I have done is set up a VPN and have certain traffic, among which the FreeNAS is included, that routes through the VPN. Now, I have set up my FreeNAS to sit on .23 and it uses the VPN and everything is cool and dandy. Furthermore, the jails are set up to use DHCP (I tried static first) which the MAC are added to the DHCP server on the pfSense box. The jails are 60 and 61.

So, the problem is that every...maybe hourish? the jails just stop getting internet. They can ping around the network but they cannot ping out to the internet. Not 8.8.8.8 or DNS. Currently I have a cron script that runs every 30 min to do:
/etc/rc.d/netif restart
dhclient epairb1
This magically fixes everything.

I have no idea what's happening or where the root cause is. The FreeNAS itself has no problems either and it's also going through the VPN as well, thus I don't think this is the problem.

Thank you for your help. I apologize if this is the wrong spot to put this, but it seemed the place to go.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Sounds like an interesting problem. Is it OpenVPN on the pfSense box?

You're saying you had the same symptoms even when the jails were not using DHCP to get IP addresses?

Do the symptoms go away when you disable the VPN?

What does "netstat -r" say in both the FreeNAS appliance, and the jail?
 

Sty

Newbie
Joined
Jun 2, 2015
Messages
16
Yes, it's OpenVPN on the pfSense.

Correct, the issues were the same either way. I merely thought that letting DHCP be handled by one device was the better option.

Unsure, I will test that later.

FreeNAS main:
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default michael.local UGS igb0
localhost link#3 UH lo0
192.168.75.0 link#1 U igb0
athena.local. link#1 UHS lo0

Internet6:
Destination Gateway Flags Netif Expire
:: localhost UGRS lo0
localhost link#3 UH lo0
::ffff:0.0.0.0 localhost UGRS lo0
fe80:: localhost UGRS lo0
fe80::%lo0 link#3 U lo0
fe80::1%lo0 link#3 UHS lo0
ff01::%lo0 localhost U lo0
ff02:: localhost UGRS lo0
ff02::%lo0 localhost U lo0

Transmission Jail:
Routing tables

Internet:
Destination Gateway Flags Netif Expire
default 192.168.75.1 UGS epair1b
localhost link#1 UH lo0
192.168.75.0 link#2 U epair1b
transmission_1 link#2 UHS lo0

Internet6:
Destination Gateway Flags Netif Expire
:: localhost UGRS lo0
localhost link#1 UH lo0
::ffff:0.0.0.0 localhost UGRS lo0
fe80:: localhost UGRS lo0
fe80::%lo0 link#1 U lo0
fe80::1%lo0 link#1 UHS lo0
ff01::%lo0 localhost U lo0
ff02:: localhost UGRS lo0
ff02::%lo0 localhost U lo0

One thing I noticed was that netstat -r was instantaneous on the FreeNAS and took a few moments on the jail.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
So, your default gateway is "michael.local". It would be better to have a hard-wired IP address in there.

In the GUI, under network->global, you can set the default gateway. Can you put in the hard-wired IP address there? All kinds of network connectivity are potentially problematic if, for example, you cannot resolve michael.local. I am no FreeBSD network master, but on every box I look at, usually the FreeNAS host itself is listed by IP address as well in that section. For example:
Code:
[root@giskard] ~# netstat -r
Routing tables

Internet:
Destination  Gateway  Flags  Netif Expire
default  192.168.1.1  UGS  lagg0
localhost  link#3  UH  lo0
192.168.1.0  link#4  U  lagg0
192.168.1.21  link#4  UHS  lo0
192.168.1.200  link#4  UHS  lo0
192.168.1.203  link#4  UHS  lo0

Internet6:
Destination  Gateway  Flags  Netif Expire
::  localhost  UGRS  lo0
localhost  link#3  UH  lo0
::ffff:0.0.0.0  localhost  UGRS  lo0
fe80::  localhost  UGRS  lo0
fe80::%lo0  link#3  U  lo0
fe80::1%lo0  link#3  UHS  lo0
ff01::%lo0  localhost  U  lo0
ff02::  localhost  UGRS  lo0
ff02::%lo0  localhost  U  lo0


(My FreeNAS host is 192.168.1.21, the .200's guys are the jails without their own VIMAGEs).

Anyway. Looks to me like network configuration within the FreeNAS is not exactly set up right, or, the FreeNAS host is not pulling correct information from whomever is sending DHCP.

I'd rather have someone like @Ericloewe though take a look because I could be full of doo doo.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
16,108
So, your default gateway is "michael.local". It would be better to have a hard-wired IP address in there.
Definitely highly dubious. I'd be surprised if it ever worked, since that field has no reason to lead to a name resolution function.
 

DrKK

FreeNAS Generalissimo
Joined
Oct 15, 2013
Messages
3,630
Well @Sty? Where are we at here sir?
 

Sty

Newbie
Joined
Jun 2, 2015
Messages
16
So, the pfSense, named michael, handles all DNS as well. It works fine, that's actually most all computers on the box.

I just double checked and the hard coded is the IP address for michael, 192.168.75.1.
 

Sty

Newbie
Joined
Jun 2, 2015
Messages
16
Even without the proxy, jails still lose connection. My "work-around" is to restart them from the server.

One thing I neglected to mention is that I switched networks recently, could that be a cause?
 
Top