SOLVED Jails and apache

[jgreco]

If you use vnet interfaces in your jails, you should be able run multiple webservers (or any service really) listening on the same ports. This is exactly what I do in some of my jails so that I can just type foo.bah in the browser. A reverse proxy would also work, and perhaps if this is intended to be externally accessible (which it sounds like it is) that might be a better (albeit more complex, another moving part) solution.

If there are down sides of avoiding the reverse proxy approach, please let me know!

There seems to be an important point missing here.

If you have a single external IP and you wish to share port :80 or :443 or whatever, then, yes, you probably want a reverse proxy of some sort.

In this case, you're probably best off creating two jails for each of the web servers, perhaps one at 192.168.1.11 and one at 192.168.1.12, so that your internal LAN has direct access, and then *another* jail at 192.168.1.10 for whatever reverse proxy engine you use. You configure port forwarding to redirect to 192.168.1.10, which then has reverse proxy forwarding directives for "site1.your.domain" to 192.168.1.11, and "site2.your.domain" to 192.168.1.12.

It's possible to do this with only two jails, but a three jail design may be a better way to design it for expandability and segregation of functionality.

I cannot tell if the massive complexity in jail configuration being discussed above is actually required because I don't use iocage (or do jails on FreeNAS for that matter), but this should actually be a very trite configuration for implementation with jails. I do extensive work with jails natively on FreeBSD and can't imagine the reason vnet would be required for this. The described scenario is literally the sort of situation that phk created jails for, and you should really only need a plain jail that has an IP alias.

 

[Oriann]

Ok I will try again with reverse proxy uf...almost 6th day
But what is the purpose of jails with vnet when it doesnt to be meant separate network stack ? I read on the freebsd docs that it will make solo jail with broadcasting IP address. Maybe I dont understand it correctly how it works.
Anyway my question should maybe point to apache forums

 

[jgreco]

Ok I will try again with reverse proxy uf...almost 6th day
But what is the purpose of jails with vnet when it doesnt to be meant separate network stack ? I read on the freebsd docs that it will make solo jail with broadcasting IP address. Maybe I don't understand it correctly how it works.
Anyway my question should maybe point to apache forums

Stepping back from FreeNAS for a moment and looking at it from a FreeBSD perspective.

Traditional jails work within the host's network stack. It seems like you understand this and it makes sense to you. If that's so, then great, you and I are on the same page.

A traditional jail works by creating an additional IP interface on the host platform. For most of the audience, this would be just adding an additional address onto the primary network interface, so if your FreeNAS is at 192.168.1.2, you might add

/etc/rc.conf:ifconfig_em0_alias0="inet 192.168.1.3 netmask 0xffffffff"

or

host# ifconfig em0 inet 192.168.1.3 netmask 0xffffffff alias

And then run a jail such as

jail /my/jail/path my.jail.name 192.168.1.3 /sbin/httpd -k start

If that makes great sense to you, fantastic, it does to me as well. In my environment, this is just a little more complicated because I make jails on the loopback interface (lo1) and advertise them into the network with OSPF. Conceptually the same but allows for layer 3 routing that provides network redundancy and resiliency benefits. Still feels like the em0 aliased jail.

At some point, though, people started to feel the limitations of using the host network stack. There are a lot of places that sell virtual servers that are just jails, and it is limiting in some ways to not be able to easily customize routing or firewall rules from within the jail. From this perspective, I understand the drive to create what is basically a better caricature of a full FreeBSD system from within the jail. We've seen jails gain support for jails within jails, mounting stuff, their own network pseudo-stack, etc.

I think this is just a little tragic because it creates a lot of unnecessary complexity.

If you thought this message was going to end on some happy "aha" note, sorry. I don't have a magic fix, but my guess from this discussion is that you have a decent grasp of the original jail implementation, and I'm just trying to say "don't feel bad. I think all this complexity is kinda tragic too."

 

[Oriann]

No problem I dont expect full solutions for my problems but just hints how to..
As you suggest then FreeNAS just tie my hands to make some thing "not too much complex" ?
So Ubuntu Server with Kubernetes or FreeBSD will be better choice to go ?

 

[jgreco]

No problem I don't expect full solutions for my problems but just hints how to..
As you suggest then FreeNAS just tie my hands to make some thing "not too much complex" ?
So Ubuntu Server with Kubernetes or FreeBSD will be better choice to go ?

I don't know. I haven't done much with the jail facility on FreeNAS because I have no use case for it, and it's quite possible you know more about it than I do. If I didn't already have way too many things to do, I'd probably go look at it and see if there was an obvious answer. Sadly, that's not in the cards right now, so we'll have to settle for what's already been discussed.

 

[Oriann]

I don't know. I haven't done much with the jail facility on FreeNAS because I have no use case for it, and it's quite possible you know more about it than I do. If I didn't already have way too many things to do, I'd probably go look at it and see if there was an obvious answer. Sadly, that's not in the cards right now, so we'll have to settle for what's already been discussed.

Ok then I need to experiment further, anyway thanks for opinions.

 

[ascl]

EDITED: @jgreco raises a really good point. A single external IP address isn't going to work with 2 instances like I was suggesting without a reverse proxy. I missed that, my apologies, I was thinking this was internal access!