How to use Openvpn & ipfw in a jail so it only connects to the VPN

[Glorious1]

Like others, I also wanted to ensure that transmission could only use the VPN. But I just couldn't get my head around the IPFW rules discussed in this thread. So I thought I'd try something a little simpler based on binding the transmission user to specific adapters. Here is what I came up with:

Code:

ipfw -f flush

ipfw -q add 00010 allow ip from any to any via tun0

ipfw -q add 00101 allow ip from me to 10.0.0.0/24 via epair0b uid transmission
ipfw -q add 00102 allow ip from 10.0.0.0/24 to me via epair0b uid transmission
ipfw -q add 00103 deny ip from any to any via epair0b uid transmission

ipfw -q add 65535 allow all from any to any

The first rule isn't really needed, but it means that I get statistics on how much the VPN is being used when i do a ipfw show, which is a nice confirmation that the VPN is being used. Note that my LAN subnet/mask is 10.0.0.0/24.

It works!

Thank you nickt! This is brilliant! I know it's been a while, but I've been banging my head against the wall for a day and a half trying to get these rules to work. Your solution is simple, and it works. I can do anything in the jail, transmission works, I can access it via the GUI remote, but when openvpn stops, transmission traffic comes to a screeching halt. Perfect!

FYI, after restarting, I found that my epair0b interface was now epair2b. So I simply removed "via epair0b" from all the above. Hopefully that will still do the job?

 

[Glorious1]

Is the guide on the first post of this topic still accurate with the latest version of FreeNAS (9.3) ?

Yes, it worked for me. But for the firewall rules see previous message.

 

[log_tugger]

Hello. I have worked through this entire tutorial. OpenVPN and ipfw seem to be working independently. The problem is when I enable both at the same time, the "tun0" interface disappears completely, which is the interface that all traffic is supposed to be routed through in order to connect to my VPN's server. I'm sure there is some config file somewhere that needs to be changed, but I have searched for hours, and I am not able to figure this out. Can anyone shed light on this situation?

 

[Glorious1]

What are your ipfw rules? I highly recommend nickt's rule set, as I mentioned a few posts back.

 

[DearestDreamer]

I'm having trouble getting this all to work. My current setup is as follows:

FreeNAS 9.3
-jail1: openvpn, 192.168.1.100
-jail2: rtorrent, 192.168.1.101

Both jails set up as VIMAGE and work as expected (openvpn client is connected and works, and the rtorrent jail also works with the webui and adding torrents etc.), but I'm confused how to apply these ipfw rules to achieve the following:

I'd like the rtorrent jail to only go through the openvpn jail, but all these tutorials i'm reading about I read that you should create the rules in each jail separately, but I thought I should have rules set on the FreeNAS host, not the jails? Wouldn't that make more sense? However this is where I'm struggling. What commands do I set in ipfw to achieve this outcome? I'd basically want to tell my torrent jail to route through the openvpn jail, while my freenas host maintains its connection directly, not through the vpn. Does anyone have a lead on this?

 

[Rudi Pittman]

I'm having trouble getting this all to work. My current setup is as follows:

FreeNAS 9.3
-jail1: openvpn, 192.168.1.100
-jail2: rtorrent, 192.168.1.101

Both jails set up as VIMAGE and work as expected (openvpn client is connected and works, and the rtorrent jail also works with the webui and adding torrents etc.), but I'm confused how to apply these ipfw rules to achieve the following:

I'd like the rtorrent jail to only go through the openvpn jail, but all these tutorials i'm reading about I read that you should create the rules in each jail separately, but I thought I should have rules set on the FreeNAS host, not the jails? Wouldn't that make more sense? However this is where I'm struggling. What commands do I set in ipfw to achieve this outcome? I'd basically want to tell my torrent jail to route through the openvpn jail, while my freenas host maintains its connection directly, not through the vpn. Does anyone have a lead on this?

I think you might need to re-read the op. The ipfw rules and openvpn are all being ran within the same jail transmission is in. It's really the only way to force the traffic through without getting really complicated. You could for instance have saznzbd and transmission running through the vpn and have couchpotato/sickrage each running in their own jail since they can still use folder watching to get files processed. As far as the rules...I've seen some really complicated ones but the ones from nickt (modified a tad) appear to be simplest:

ipfw_rules file from /media (the rules can be edited from the nas this way) These rules basically force transmission (you would have to change them to the uid for rtorrent) to use the tun0 device for traffic while still allowing normal local lan traffic to/from transmission. My lan is on 192.168.3.0. The last rule originally had 65535 as the number to add but that caused an error so I dropped it to 65534. The 3 lines in the middle could be repeated and modified for each app you want to force to use the vpn. You may also have to compile your own version of openvpn (see the previous instructions) because the default one you get from "pkg install openvpn" does not have the feature enabled where userid/password can be stored in a file which is required if you are going to have the vpn autostart via rc.conf.

Code:

add 00010 allow ip from any to any via tun0
  
add 00101 allow ip from me to 192.168.3.0/24 uid transmission
add 00102 allow ip from 192.168.3.0/24 to me uid transmission
add 00103 deny ip from any to any uid transmission
  
add 65534 allow all from any to any

 

[Ash Swainson]

Then copy over the certs and keys

Code:

[root@transmission_1 /]# cp /media/ca.crt /usr/local/etc/openvpn/keys/ca.crt
[root@transmission_1 /]# cp /media/user.crt /usr/local/etc/openvpn/keys/user.crt
[root@transmission_1 /]# cp /media/user.key /usr/local/etc/openvpn/keys/user.key
[root@transmission_1 /]# cp /media/ta.key /usr/local/etc/openvpn/keys/ta.key

hi guys, I'm relatively new to freenas and command line programming in general. I have given your tutorial a go but when i go to test the connection using ifconfig there is no non-local connection. The only thing I can see i did different was I only had 1 key and certificate, provided from my VPN service. but as above it implies there are 2? I have seen other tutorials that have a step about "generating certificates", but am having trouble understanding it as when i try execute their tutorials myself i get errors about missing directories (I think its an old tutorial on an old version)

any help would be much appricated guys

Regards
Ash

 

[Glorious1]

hi guys, I'm relatively new to freenas and command line programming in general. I have given your tutorial a go but when i go to test the connection using ifconfig there is no non-local connection. The only thing I can see i did different was I only had 1 key and certificate, provided from my VPN service. but as above it implies there are 2? I have seen other tutorials that have a step about "generating certificates", but am having trouble understanding it as when i try execute their tutorials myself i get errors about missing directories (I think its an old tutorial on an old version)

I also had only one certificate, and no "keys", and it works. The location for the certs/keys has to match what is in the configuration files. For me (with Private Internet Access) all the files had
ca /usr/local/etc/openvpn/ca.crt
so that's where I put the ca.crt, not in a keys folder.
I also have a crl.pem file which may be like a key, and it is in the same place as stated in the config file:
crl-verify /usr/local/etc/openvpn/crl.pem
Make sure your location matches that specified in the configuration file.

 

[Ash Swainson]

I also had only one certificate, and no "keys", and it works. The location for the certs/keys has to match what is in the configuration files. For me (with Private Internet Access) all the files had
ca /usr/local/etc/openvpn/ca.crt
so that's where I put the ca.crt, not in a keys folder.
I also have a crl.pem file which may be like a key, and it is in the same place as stated in the config file:
crl-verify /usr/local/etc/openvpn/crl.pem
Make sure your location matches that specified in the configuration file.

Hi,

Thanks for your response. Was your crl.pem file provided by your vpn provider? I have double checked the paths to the certificate and key and they are correct, i even tried changing them to force them to be wrong and it caused an error so I am pretty sure they arent the problem.

 

[Glorious1]

Was your crl.pem file provided by your vpn provider?

Yes. Hard to know what your problem is. You said "when i go to test the connection using ifconfig there is no non-local connection." I'm not sure what that means. You can check if things are working right using a special torrent from http://checkmytorrentip.net/ and see what if the IP is your internet provider's or the VPN server's.

 

[Ash Swainson]

Yes. Hard to know what your problem is. You said "when i go to test the connection using ifconfig there is no non-local connection." I'm not sure what that means. You can check if things are working right using a special torrent from http://checkmytorrentip.net/ and see what if the IP is your internet provider's or the VPN server's.

My understanding is that when I run ipconfig I should get a local address and one assigned by the VPN (not in my 192.168.x.x subnet) but i dont. I think I should be seeing a tun0 connection

 

[Glorious1]

Probably so. Sometimes when there is trouble, you can stop, then start the firewall (if you set one up), then do the same with openvpn. For me it is with these commands:
To start or stop the firewall: /etc/rc.d/ipfw start/stop
To start or stop openvpn: /usr/local/etc/rc.d/openvpn start/stop

 

[Ash Swainson]

Probably so. Sometimes when there is trouble, you can stop, then start the firewall (if you set one up), then do the same with openvpn. For me it is with these commands:
To start or stop the firewall: /etc/rc.d/ipfw start/stop
To start or stop openvpn: /usr/local/etc/rc.d/openvpn start/stop

Ok so i have made some progresss, I can get the VPN to connect, but it doesnt seem to be working entirely as expected

When i reboot my whole freenas system, the transmission plug-in will remain in the off position and wont turn on until i restart the jail. at that point transmission automatically restarts but i have to manually start the openvpn service to get it to work. not sure what could be causing this? It must be related to the addition of the openvpn pkg as this never happened with transmission before.

Any ideas?

TIA for any help guys :)

 

[windyboi]

Hi,

I configured as per the instructions combined with the new shorter rules on page 7, but I cannot seem to get the google.com ping and kill openvpn test to work.. Any ideas what might be the problem? no errors on openvpn start.

I have the following added to my rc.conf:
openvpn_enable="YES"
openvpn_configfile="/usr/local/etc/openvpn/openvpn.conf"
firewall_enable="YES"
firewall_type="/media/ipfw_rules"

But, after I reboot the system and run ipfw_list I get a completely new set of rules that I've never seen before.. Any ideas??

[root@transmission_1 /]# ipfw list
00100 allow ip from any to any via lo0
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00400 deny ip from any to ::1
00500 deny ip from ::1 to any
00600 allow ipv6-icmp from :: to ff02::/16
00700 allow ipv6-icmp from fe80::/10 to fe80::/10
00800 allow ipv6-icmp from fe80::/10 to ff02::/16
00900 allow ipv6-icmp from any to any ip6 icmp6types 1
01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136
65535 allow ip from any to any

How can I test that the openvpn client is actually working first before I look to troubleshoot the firewall problem? Thanks


Here is the contents of me ipfw_rules, ifconfig and openvpn.conf:

[root@transmission_1 /]# more /media/ipfw_rules
ipfw -f flush
ipfw -q add 00010 allow ip from any to any via tun0
ipfw -q add 00101 allow ip from me to 192.168.1.0/24 via epair2b uid transmission
ipfw -q add 00102 allow ip from 192.168.1.0/24 to me via epair2b uid transmission
ipfw -q add 00103 deny ip from any to any via epair2b uid transmission
ipfw -q add 65535 allow all from any to any

[root@transmission_1 /]# ifconfig
epair2b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8<VLAN_MTU>
ether 02:f9:8c:00:0b:0b
inet 192.168.1.4 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
status: active

tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
inet 10.199.1.6 --> 10.199.1.5 netmask 0xffffffff
nd6 options=1<PERFORMNUD>
Opened by PID 11427


[root@transmission_1 /]# more /usr/local/etc/openvpn/openvpn.conf
client
dev tun
proto udp
remote nl.privateinternetaccess.com 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
tls-client
remote-cert-tls server
auth-user-pass pass.txt
comp-lzo
verb 1
reneg-sec 0
crl-verify crl.pem

 

[Rudi Pittman]

Here are the rules that are working fine for me..compare them to yours (right off I see I'm not doing the flush that you are nor am I putting ipfw -q in front which is only needed to run a line manually):

Code:

add 00010 allow ip from any to any via tun0

add 00101 allow ip from me to 192.168.3.0/24 uid transmission
add 00102 allow ip from 192.168.3.0/24 to me uid transmission
add 00103 deny ip from any to any uid transmission

add 01000 allow log udp from 192.168.0.0/16 to 208.67.222.222 dst-port 53 keep-state
#add 01002 allow log udp from 192.168.0.0/16 to 10.4.0.1 dst-port 53 keep-state
add 01004 allow log udp from 192.168.0.0/16 to 208.67.220.220 dst-port 53 keep-state

add 65534 allow all from any to any
#using 65535 caused an error but 65534 did not

You can "tail -f /var/log/messages" and watch the openvpn handshake....it should end with something similar to:

Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/ifconfig tun0 192.168.149.38 192.168.149.1 mtu 1500 netmask 255.255.255.0 up
Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/route add -net 192.168.149.0 192.168.149.38 255.255.255.0
Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/route add -net 162.253.128.67 192.168.3.1 255.255.255.255
Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/route delete -net 0.0.0.0 192.168.3.1 0.0.0.0
Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/route add -net 0.0.0.0 192.168.149.1 0.0.0.0
Feb 17 02:36:49 transmission_1 openvpn[70323]: Initialization Sequence Completed

You should do an ifconfig and verify your setup is using tun0...on mine it was using tun2 until I forced it to use tun0 via a command in the config...the rules won't apply if it's on a different tun device. My final "ipfw list" rules (I could eliminate the duplicate last 2 but it's working so not a high priority):

[root@transmission_1 /media]# ipfw list
00010 allow ip from any to any via tun0
00100 allow ip from any to any via lo0
00101 allow ip from me to 192.168.3.0/24 uid transmission
00102 allow ip from 192.168.3.0/24 to me uid transmission
00103 deny ip from any to any uid transmission
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00400 deny ip from any to ::1
00500 deny ip from ::1 to any
00600 allow ipv6-icmp from :: to ff02::/16
00700 allow ipv6-icmp from fe80::/10 to fe80::/10
00800 allow ipv6-icmp from fe80::/10 to ff02::/16
00900 allow ipv6-icmp from any to any ip6 icmp6types 1
01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136
01000 allow log udp from 192.168.0.0/16 to 208.67.222.222 dst-port 53 keep-state
01004 allow log udp from 192.168.0.0/16 to 208.67.220.220 dst-port 53 keep-state
65534 allow ip from any to any
65535 allow ip from any to any

 

[Glorious1]

. . .
But, after I reboot the system and run ipfw_list I get a completely new set of rules that I've never seen before.. Any ideas??
. . . .
How can I test that the openvpn client is actually working first before I look to troubleshoot the firewall problem?

Yes, your rules don't look quite like @nickt's rules which are working fine for me:
https://forums.freenas.org/index.php?goto/post&id=191684#post-191684

I think there may be more than one rc.conf - did you edit /conf/base/etc/rc.conf? And that may have to be redone after OS update, as these are not stored in saved configuration.

The test I use to see if openvpn is working is to use a special torrent from one or both of these sites that report what IP they are on. If it's not your external IP, it's working. You can do a geo-ip lookup to see where the IP is and see if it matches the server in your config file.
http://checkmytorrentip.net/
http://ipmagnet.services.cbcdn.com

 

[wah00kid]

Here are the rules that are working fine for me..compare them to yours (right off I see I'm not doing the flush that you are nor am I putting ipfw -q in front which is only needed to run a line manually):

Code:

add 00010 allow ip from any to any via tun0

add 00101 allow ip from me to 192.168.3.0/24 uid transmission
add 00102 allow ip from 192.168.3.0/24 to me uid transmission
add 00103 deny ip from any to any uid transmission

add 01000 allow log udp from 192.168.0.0/16 to 208.67.222.222 dst-port 53 keep-state
#add 01002 allow log udp from 192.168.0.0/16 to 10.4.0.1 dst-port 53 keep-state
add 01004 allow log udp from 192.168.0.0/16 to 208.67.220.220 dst-port 53 keep-state

add 65534 allow all from any to any
#using 65535 caused an error but 65534 did not

You can "tail -f /var/log/messages" and watch the openvpn handshake....it should end with something similar to:

Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/ifconfig tun0 192.168.149.38 192.168.149.1 mtu 1500 netmask 255.255.255.0 up
Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/route add -net 192.168.149.0 192.168.149.38 255.255.255.0
Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/route add -net 162.253.128.67 192.168.3.1 255.255.255.255
Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/route delete -net 0.0.0.0 192.168.3.1 0.0.0.0
Feb 17 02:36:49 transmission_1 openvpn[70323]: /sbin/route add -net 0.0.0.0 192.168.149.1 0.0.0.0
Feb 17 02:36:49 transmission_1 openvpn[70323]: Initialization Sequence Completed

You should do an ifconfig and verify your setup is using tun0...on mine it was using tun2 until I forced it to use tun0 via a command in the config...the rules won't apply if it's on a different tun device. My final "ipfw list" rules (I could eliminate the duplicate last 2 but it's working so not a high priority):

[root@transmission_1 /media]# ipfw list
00010 allow ip from any to any via tun0
00100 allow ip from any to any via lo0
00101 allow ip from me to 192.168.3.0/24 uid transmission
00102 allow ip from 192.168.3.0/24 to me uid transmission
00103 deny ip from any to any uid transmission
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00400 deny ip from any to ::1
00500 deny ip from ::1 to any
00600 allow ipv6-icmp from :: to ff02::/16
00700 allow ipv6-icmp from fe80::/10 to fe80::/10
00800 allow ipv6-icmp from fe80::/10 to ff02::/16
00900 allow ipv6-icmp from any to any ip6 icmp6types 1
01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136
01000 allow log udp from 192.168.0.0/16 to 208.67.222.222 dst-port 53 keep-state
01004 allow log udp from 192.168.0.0/16 to 208.67.220.220 dst-port 53 keep-state
65534 allow ip from any to any
65535 allow ip from any to any

Hey Rudi! Thanks for picking up the OP and Nickt

If you wouldnt mind I have a couple of questions mostly in the noob category, anyway:

My openvpn is going strong and I'm trying to add in this firewall settings.

You say in #170 that your LAN is 192.168.3.0 ... is this the same as the local address of your Freenas GUI, your transmission jail, or from your router/DHCP table???

Also 1000 and 1004 seem like your DNS servers.. those are public ones, from your ISP, from your VPN?? Can I just use google DNSs?

Finally from the OP the recommended way to test this would be to run an openvpn stop and then ping google.com and traffic should be denied... is this still the best way to test the firewall with the new rules you use? I guess, how can I make sure this is working since my openvpn is already going strong and I can test it using the CheckMyTorrentIp?

wooo yeah ended up getting it with this setup:

https://www.reddit.com/r/freenas/comments/41fhz3/configuration_guide_for_openvpn_and_ipfw_so_that/

 

[windyboi]

Yes, your rules don't look quite like @nickt's rules which are working fine for me:
https://forums.freenas.org/index.php?goto/post&id=191684#post-191684

I think there may be more than one rc.conf - did you edit /conf/base/etc/rc.conf? And that may have to be redone after OS update, as these are not stored in saved configuration.

The test I use to see if openvpn is working is to use a special torrent from one or both of these sites that report what IP they are on. If it's not your external IP, it's working. You can do a geo-ip lookup to see where the IP is and see if it matches the server in your config file.
http://checkmytorrentip.net/
http://ipmagnet.services.cbcdn.com

I edited /etc/rc.conf - is that correct? or do I need to edit /conf/base/etc/rc.conf also
My rules are exactly the same as the rules nickt posted (except I changed my epair0b to epair2b to reflect my setup) or am I missing something?

Thanks, I'll check those links once I've got my ipfw working.

_________

Here are the rules that are working fine for me..compare them to yours (right off I see I'm not doing the flush that you are nor am I putting ipfw -q in front which is only needed to run a line manually):

Code:

add 00010 allow ip from any to any via tun0

add 00101 allow ip from me to 192.168.3.0/24 uid transmission
add 00102 allow ip from 192.168.3.0/24 to me uid transmission
add 00103 deny ip from any to any uid transmission

add 01000 allow log udp from 192.168.0.0/16 to 208.67.222.222 dst-port 53 keep-state
#add 01002 allow log udp from 192.168.0.0/16 to 10.4.0.1 dst-port 53 keep-state
add 01004 allow log udp from 192.168.0.0/16 to 208.67.220.220 dst-port 53 keep-state

add 65534 allow all from any to any
#using 65535 caused an error but 65534 did not

Your rules look a lot different to the ones in post 7. What are the addresses 208.67.222.222, 208.67.220.220 ?

I did verify tun0 is being used already.
Thank you


Edit: So I can verify that my openvpn IS set up correctly, but I just can't seem to get the "ping google.com and stop openvpn" test to work

 

[Glorious1]

I edited /etc/rc.conf - is that correct? or do I need to edit /conf/base/etc/rc.conf also

Yes, I can't remember where I learned it, but you have to change the second one to survive reboot - exactly the problem you reported earlier.

 

[windyboi]

Yes, I can't remember where I learned it, but you have to change the second one to survive reboot - exactly the problem you reported earlier.

There isn't a file that exists with this name :/

UPDATE: I have made some progress by using the post described above, https://www.reddit.com/r/freenas/comments/41fhz3/configuration_guide_for_openvpn_and_ipfw_so_that/

major change I made was to use firewall_script="" instead of firewall_type, and I made a couple of modifications to my ip.fw file and now ipfw list gives me some good stuff, but the google ping test STILL does not work!!! can anyone please help shed some light on this?

[root@transmission_1 /]# cat /media/ipfw.rules
#!/bin/bash
# Flush out the list before we begin
ipfw -q -f flush
# Set rules command prefix
cmd="ipfw -q add"
vpn="tun0"
lan="epair2b"
# allow any connection to/from VPN interface
$cmd 00010 allow all from any to any via $vpn
# allow connection to/from LAN by Transmission
$cmd 00101 allow all from me to 192.168.1.0/24 via $lan uid transmission
$cmd 00102 allow all from 192.168.1.0/24 to me via $lan uid transmission
# deny any Transmission connection outside LAN that does not use VPN
$cmd 00103 deny all from any to any via $lan uid transmission

[root@transmission_1 /]# ipfw list
00010 allow ip from any to any via tun0
00101 allow ip from me to 192.168.1.0/24 via epair2b uid transmission
00102 allow ip from 192.168.1.0/24 to me via epair2b uid transmission
00103 deny ip from any to any via epair2b uid transmission
65535 allow ip from any to any

Thanks