Which is better? allin one jail or multiple jails for multiple tasks?

[Robert Thomspon]

Hey gents, was wondering if anyone might have an idea on best setup for multiple related apps installed in the same jail and if there is a benefit to doing so or if its better to have individual jails for each plugin?
Basically, im getting ready to do a clean install of the 11.3 branch and figured i'd rebuild my jails at that time...
Is it better to say, stick radarr, lidarr, sonarr, mySQL, and SabNZBD into one jail, transmission and openVPN (for it to run behind a VPN) in another, and nextcloud into a third (i also run zoneminder but assume that that is resource intensive and read/write intensive enough on its own to warrant its own jail... i think)...

OR:
Should I create separate and distinct jails for each plugin/app?

And if one setup is better than the other, is there a layman's explanation as to why?

Thanks in advance for any input.

 

[sretalla]

Should I create separate and distinct jails for each plugin/app?

Separate

And if one setup is better than the other, is there a layman's explanation as to why?

Components like mono, perl, python, php...

Different plugins/apps are at different levels in terms of what they need/are tested with as versions go forward... but they are developed by different people at different speeds, so you can't be guaranteed that, for example, sonarr and radarr will both be happy with the same version of mono when one of them wants to change it.

Keeping them separate doesn't cost much in terms of disk space nor RAM and CPU, so there's little argument for the monolithic do-all jail.

 

[jgreco]

Separate is better but can be more work.

Jails are similar in many ways to Linux containers (and predate them, and are much better IMO). You get the advantage of compartmentalization without the need to have a separate virtual machine.

@sretalla gives some good reasons, from a developer's point of view. From the '80's to around 2010, I spent my life (as did most other UNIX admins) working on physical hosts. The base OS (FreeBSD in this case, but Linux is an even more stellar example) will generally have a mechanism used to install ports and keep things up to date, but these things tend to be fragile when you have more than a handful installed, and a big server that's doing database, web, mail, DNS, and lots of other stuff can suddenly find a bunch of services crashed when you update something like OpenSSL, and you can even get into unresolvable situations because one thing REALLY needs OpenSSL 1.0 while another no longer works with OpenSSL 1.0. As an example.

I've worked on a lot of trainwreck-grade systems where people just kept installing service after service on the base platform, and things keep getting more fragile if you do it that way. One day you upgrade one component and get a cascading failure of lots of stuff.

Jails allow you to create an isolated environment that has just the dependencies you need for providing a given service. You can install your Apache/PHP/MySQL/OpenSSL 1.2 jail for web service while also installing Postfix/Dovecot/MariaDB/OpenSSL 1.0 in a different jail for mail service - no longer limiting yourself to making a host-level choice between installing MySQL and MariaDB, or OpenSSL 1.0 and 1.2. Doing things this way makes it more manageable and easier to do comprehensive testing of components as you update them.

I've now been working with jails basically since PHK committed the code, about 20 years. You can take the technology quite far, such as creating Internet-facing services with a skeleton layout that doesn't even include /bin/sh, which helps keep the skript kiddiez out. See, there's no reason you *must* include an entire OS image in there, so if you've got a server load balancer, the jail tree should just contain haproxy and its configuration files. But that's relatively advanced stuff (you can get some sense of some of my work here).

For newcomers, it's still great to have a standard FreeBSD tree within a jail and use ports to install the things you need for that specific jail.

 

[Robert Thomspon]

Thanks gents... that info is good to know... and applicable to my setup already (i had had my MySQL that keeps kodi in line on the same jail as my sonarr/sabnzbd/radarr and it took a dump one day... not sure what caused it but i wasnt able to access the jail at all (well, i could, but not as a user capable of doing anything)... which made me reach out for some help to figure out how to regain access to the jail... all is well now and the SQL server is kept seperately from those services... but its good to know that seperating them (because of the use of things like mono) is a good idea... ive been building 1 do-all jail for the day to day crap that the server does (TV, movies, audio...etc) and seperating more crucial things (like next cloud) in case of a fault... (it doesnt hurt too much if sonarr and the like go down... whats the worst that happens? i miss the shitty end og game of thrones? :) ... but if nextcloud goes down, and the wife cant access pics/videos of the kid, there's hell to pay)

Thanks again for the advice gents...

 

[jgreco]

multiple related apps installed in the same jail

Oh, and I meant to comment on this. Depends on how "related" they are and how much work you're willing to go to.

Some stuff, such as an Apache/PHP jail, necessarily have some components that have to be in the same jail. But you *could* put your SQL server in there if you wanted to think of the jail as "my webserver jail" or you *could* put the SQL server as a separate jail because sometimes there's stupid amounts of drama doing updates of components, or a major update of the SQL version. My tendency is to separate functionality that does not have an inherent need to be tied together. If you separate out the SQL, for example, you can make a tarball of the "working" SQL, sub in a new jail, and either it works or you quickly roll back. If you put the SQL in with the web server, it's harder. That's the best working example I can think of for what I'm talking about.

 

[castle]

I create a separate jail for each plugin, I even do this for virtual machines.

I think it's a lot more tidy to have some sort of separation, done get incompatible dependencies etc. thus today I have:

Jails:

• mosquitto
• nodered
• grafana
• influx
• minos
• plex

and virtual machines (debian10):

• Unifi
• pihole
• motioneye

All hosted on a two ssd's in Raid-1.