Setting up a reverse proxy in a jail with nginx

protoss_666 · Jun 13, 2020

Hello, I have some questions.

On a VM mounted on virtualbox, I have FreeNAS installed.
This VM has a bridge configuration to take internet from my home network.
I have created a jail, there I am configuring a reverse proxy to attend to all incoming requests to my freeNAS.
My idea is to install a SSL Lets encrypt wilcard certificate over the jail with nginx. As investigated in:

Challenge Types - Let's Encrypt - Certificados SSL/TLS Gratuitos

Cuando solicita un certificado a Let’s Encrypt, nuestros servidores comprueba que usted está autorizado o controla los nombres de dominio en ese certificado utilizando “desafíos”, según lo define el estándar ACME. La mayoría de las veces, esta validación es manejada automáticamente por su...

letsencrypt.org

To be able to do it, lets encrypt says that I have to do the validation by DNS and I have to create some TXT records and the DNS zone has to be accessible via API. They display a list of supported DNS services:
In my case I plan to use Cloudflare.

My # 1 question is:
The proxy must be assigned a public IP so that it can resolve the DNS, but the jail has a local IP configured. In the jail I have a VNET + NAT configured without DHCP (fixed local IP).
Should I use a Dynamic DNS service to be able to link my dynamic IP (from the ISP) with the local IP of the jail and then do a port forwarding on my router? The problem I am having is that the jail is under another subnet, the Jail IP is 172.6.0.2.
When I want to configure port forwarding on my router with IP 172.6.0.2 it gives me the error that it is not on the same network.

My # 2 question is:
In the jail where I have the reverse proxy, how can I link my domain? What steps should I take?

My # 3 question is:
My FreeNAS private IP is 192.168.0.105 (NAT)
My Jail's private IP (r-proxy) is 172.6.0.2 (NAT)

If I ping from my PC to the jail, I cannot access it.

You can help?

Cheers

sretalla · Jun 14, 2020

protoss_666 said:
The proxy must be assigned a public IP so that it can resolve the DNS

That just means you need to forward the appropriate port(s)... probably 80 and/or 443... on your router/firewall (with your public IP address) to the jail's private IP and those packets must get there over NAT or whatever you have done to yourself there.

protoss_666 · Jun 25, 2020

I have successfully installed the letsencrypt certificate with certbot in my reverse-proxy with nginx in a jail in FreeNAS with the -manual method (I am not using the cloudflare plugin because now the API is not accessible for free accounts).

Code:

Press Enter to Continue
Waiting for verification ...
Cleaning up challenges

IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
   /usr/local/etc/letsencrypt/live/r-proxy.nas.ethopolis.tech/fullchain.pem
   Your key file has been saved at:
   /usr/local/etc/letsencrypt/live/r-proxy.nas.ethopolis.tech/privkey.pem
   Your cert will expire on 2020-09-22. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew * all * of your certificates, run
   "certbot renew"
- If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
   Donating to EFF: https://eff.org/donate-le

The problem I am having is that when I run the command:

Code:

openssl s_client -connect r-proxy.nas.ethopolis.tech:443

I get the message "gethostbyname failure":

Code:

root@reverse-proxy:~ # openssl s_client -connect r-proxy.nas.ethopolis.tech:443
gethostbyname failure
connect:errno=0

I have configured my nginx.conf from jail as follows:

Code:

#user  nobody;
worker_processes  1;

# This default error log path is compiled-in to make sure configuration parsing
# errors are logged somewhere, especially during unattended boot when stderr
# isn't normally logged anywhere. This path will be touched on every nginx
# start regardless of error log location configured here. See
# https://trac.nginx.org/nginx/ticket/147 for more info.
#
#error_log  /var/log/nginx/error.log;
#

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    server {
        listen       80;
        server_name  r-proxy.nas.ethopolis.tech;
        return 301 https://$server_name$request_uri;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   /usr/local/www/nginx;
            index  index.html index.htm;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/local/www/nginx-dist;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }

    #    listen       8000;
    #    listen       somename:8080;
    #    server_name  somename  alias  another.alias;

    #    location / {
    #        root   html;
    #        index  index.html index.htm;
    #    }
    #}


    # HTTPS server
    #
    server {
        listen       443 ssl;
        server_name  r-proxy.nas.ethopolis.tech;

        ssl_certificate     /usr/local/etc/letsencrypt/live/r-proxy.nas.ethopol
is.tech/fullchain.pem;
        ssl_certificate_key /usr/local/etc/letsencrypt/live/r-proxy.nas.ethopol
is.tech/privkey.pem;

        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  10m;

        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;

        location / {
            root   html;
            index  index.html index.htm;
        }
    }

}

But by executing the following command, I get this result. We see that both port 80 and 443 are open::

Code:

root@reverse-proxy:~ # netstat -an -p tcp| grep LISTEN
tcp4       0      0 *.443                  *.*                    LISTEN
tcp4       0      0 *.80                   *.*                    LISTEN
root@reverse-proxy:~ # sockstat -4
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
www      nginx      3122  7  tcp4   *:80                  *:*
www      nginx      3122  8  tcp4   *:443                 *:*
root     nginx      3121  7  tcp4   *:80                  *:*
root     nginx      3121  8  tcp4   *:443                 *:*

I suspect the problem has to do with the CNAME setting (r-proxy.nas.ethopolis.tech) pointing to a Dynamic DNS of NO-IP. Therefore, when executing this CNAME, the freeNAS general interface is executed, when the correct thing would be to try to access the created jail.

What could be the problem?

My jail's IP is 127.16.xxx.xxx (NAT), it is different from FreeNAS's local IP (192.168.xxx.xxx). I understand that it is not possible to access the jail from the outside.

In my router I can only define port forwarding to my FreeNAS with 192.168.xxx.xxx, I cannot do it with the IP of jail 127.xxx.xxx.xxx (NAT).
What should I configure? In FreeNAS should I internally configure a forwarding from 192.168.xxx.xxx/24> 127.16.xxx.xxx/30?

What advice can you give me?

Regards

sretalla · Jun 26, 2020

protoss_666 said:
What could be the problem?

My jail's IP is 127.16.xxx.xxx (NAT), it is different from FreeNAS's local IP (192.168.xxx.xxx). I understand that it is not possible to access the jail from the outside.

Question and answer already there...

You can't really run a reverse proxy in a way that it isn't accessible from the outside of the box (or it can't do its job).

protoss_666 said:
In my router I can only define port forwarding to my FreeNAS with 192.168.xxx.xxx, I cannot do it with the IP of jail 127.xxx.xxx.xxx (NAT).
What should I configure? In FreeNAS should I internally configure a forwarding from 192.168.xxx.xxx/24> 127.16.xxx.xxx/30?

You should configure the jail to use VNET and get its own IP address on the 192.168... subnet, then forward the ports to it.

protoss_666 · Jul 1, 2020

I am trying to configure a jail in freenas inside virtualbox.

The problem I am having is that I cannot access the internet inside my jail.
My jail is configured without NAT. Only with VNET without DHCP.
My jail's network settings are:

Code:

root@reverse-proxy:~ # ifconfig

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384

        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>

        inet6 ::1 prefixlen 128

        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1

        inet 127.0.0.1 netmask 0xff000000

        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

        groups: lo

epair0b: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

        options=8<VLAN_MTU>

        ether 0a:00:27:b5:7c:2b

        hwaddr 02:4d:d5:00:05:0b

        inet 192.168.0.105 netmask 0xffffff00 broadcast 192.168.0.255

        inet 0.0.0.0 netmask 0xff000000 broadcast 255.255.255.255

        inet6 fe80::800:27ff:feb5:7c2b%epair0b prefixlen 64 scopeid 0x2

        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)

        status: active

        groups: epair

My FreeNAS network configuration is:

Code:

root@freenas[~]# ifconfig

le0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500

        description: 192.168.0.103

        options=8<VLAN_MTU>

        ether 08:00:27:53:ea:5d

        hwaddr 08:00:27:53:ea:5d

        inet 192.168.0.103 netmask 0xffffff00 broadcast 192.168.0.255

        nd6 options=9<PERFORMNUD,IFDISABLED>

        media: Ethernet autoselect

        status: active

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384

        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>

        inet6 ::1 prefixlen 128

        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2

        inet 127.0.0.1 netmask 0xff000000

        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

        groups: lo

bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500

        ether 02:5a:f8:c5:85:00

        nd6 options=1<PERFORMNUD>

        groups: bridge

        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15

        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200

        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0

        member: vnet0.1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>

                ifmaxaddr 0 port 4 priority 128 path cost 2000

        member: le0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>

                ifmaxaddr 0 port 1 priority 128 path cost 2000000

vnet0.1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500

        description: associated with jail: r-proxy as nic: epair0b

        options=8<VLAN_MTU>

        ether 0a:00:27:b5:7c:2a

        hwaddr 02:35:d0:00:04:0a

        inet6 fe80::800:27ff:feb5:7c2a%vnet0.1 prefixlen 64 scopeid 0x4

        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>

        media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)

        status: active

        groups: epair

Executing the netstat -nr command, I have the following table:

Code:

root@reverse-proxy:~ # netstat -nr
Routing tables

Internet:

Destination        Gateway        Flags     Netif Expire
0.0.0.0/8          link#2         U         epair0b
default            192.168.0.1    UGS       epair0b
127.0.0.1          link#1         UH        lo0
192.168.0.0/24     link#2         U         epair0b
192.168.0.105      link#2         UHS       lo0


Internet6:

Destination                        Gateway                   Flags   Netif      Expire
::/96                               ::1                      UGRS    lo0
::1                                 link#1                   UH      lo0
::ffff:0.0.0.0/96                   ::1                      UGRS    lo0
fe80::/10                           ::1                      UGRS    lo0
fe80::%lo0/64                       link#1                   U       lo0
fe80::1%lo0                         link#1                   UHS     lo0
fe80::%epair0b/64                   link#2                   U       epair0b
fe80::800:27ff:feb5:7c2b% epair0b   link#2                   UHS     lo0
ff02::/16                           ::1                      UGRS    lo0

root@reverse-proxy:~ #

In the virtualbox network configuration I have configured a bridge and the promiscuous mode> allow everything.

What could be the problem?.

Regards

Samuel Tai · Jul 1, 2020

Do you have a default gateway inside the jail? Check netstat -rn. If not, this is set in the jail's Basic Properties. Also, make sure your /etc/resolv.conf inside the jail is also configured with DNS settings.

protoss_666 · Jul 2, 2020

Hello,

Code:

root@reverse-proxy:~ # netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
0.0.0.0/8          link#2             U       epair0b
default            192.168.0.1        UGS     epair0b
127.0.0.1          link#1             UH          lo0
192.168.0.0/24     link#2             U       epair0b
192.168.0.105      link#2             UHS         lo0

Internet6:
Destination                       Gateway                       Flags     NetifExpire
::/96                             ::1                           UGRS        lo0
::1                               link#1                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%lo0/64                     link#1                        U           lo0
fe80::1%lo0                       link#1                        UHS         lo0
fe80::%epair0b/64                 link#2                        U       epair0b
fe80::800:27ff:feb5:7c2b%epair0b  link#2                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0
root@reverse-proxy:~ #

I have configured the default gateway in "basic properties":

My /etc/resolv.conf (jail):

Code:

# Generated by resolvconf
search d.nas.ethopolis.tech
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 1.1.1.1

Important Announcement for The TrueNAS Community.

Setting up a reverse proxy in a jail with nginx

protoss_666

Cadet

Challenge Types - Let's Encrypt - Certificados SSL/TLS Gratuitos

sretalla

Powered by Neutrality

protoss_666

Cadet

sretalla

Powered by Neutrality

protoss_666

Cadet

Samuel Tai

Never underestimate your own stupidity

protoss_666

Cadet

Similar threads

Important Announcement for The TrueNAS Community.