Should I stop using jails?

[mow4cash]

When I began using FreeNAS I was attracted to the easy use of plugins. Then I discovered plugins are not that great so I switched to straight jails. Jump forward to Corral and I realize Docker is great but Corral is unstable. Now I'm using jails that are going to have to be converted to iocage and RancherOS that stopped working on me. FreeBSD is great but seems like a very poor choice for running most apps.

Should I start using FreeNAS for storage only and save up to build another server for apps? What should I run on the server, ESXi with a Linux distro for Docker? Do you create multiple VM to suit the needs for each app or groups of apps? I'm trying to get feel for how to do this right because I feel like I'm doing this all wrong.

 

[dlavigne]

Notes that iocage is jails, just made using a newer backend. Jails are also much lighter weight resource-wise than VMs or other virtualization solutions.

 

[joeschmuck]

You are not the only person debating the best way forward. My advice is to stick wiht the jails for right now and wait until FreeNAS 11.2 comes out, then evaluate what options you have available. It may make more sense to continue using a jail for you, or maybe a Linux VM (under FreeNAS bhyve) and run everything on that. There are many ways a person can go here but unfortunately it's just too early to tell how things will really work until the product is available to use.

 

[Chris Moore]

I am using ESXi to run all my virtual machines since FreeNAS took away PHP VirtualBox. The only thing that I still use FreeNAS for is the Plex Plug-in Jail and storage.
If the new virtualization platform becomes stable under FreeNAS, I am willing to give it a try. I just couldn't wait for it.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Alecmascot]

Wait until FreeNAS 11.2 comes out, then evaluate what options you have available.

I am watching this too.
I am currently on 11.0-U4 and with the number of "features" being added I will wait until 11.2-U1 or even 11.2-U2.

 

[danb35]

In general, IMO, anything you're going to do on the FreeNAS box that can be done well in a jail, should be done in a jail. They have worked well for a long time, and continue to do so. They're simpler to administer tham VMs, and the overhead is much lower. If you're setting up a new jail, you might as well do it with iocage, as that's the direction FreeNAS is moving in.

Lots of people went all googly-eyed over Docker support in The Release That Must Not Be Named. No doubt there are great things about Docker, but I haven't seen them yet. But the requirement that they run in a Linux VM adds both overhead and resource limitations, neither of which seem like a good thing. And it also makes it more complicated to expose storage to those apps, which is usually needed.

If you just have to run a different OS, things get more complicated. I haven't played with bhyve much (or with ESXi at all), but the latter seems much better-supported.

 

[scrappy]

I am sticking with jails whenever and wherever it makes sense to do so. If it makes more sense to run something in Docker or Linux I have no problem going that route. Yes, FreeNAS Corral was a black eye to what could've been amazing. I was very torn last year whether or not to abandon FreeNAS altogether, but the FreeNAS team has worked hard to make FreeNAS 11/.1 a solid release and overall, FreeNAS offers more of what I want in a server platform than any other option currently available IMHO.

 

[mow4cash]

Thanks everyone for the feedback. I will wait for 11.2 to see where things are headed. So pretty much I should hope FreeNAS creates a solid virtualization platform. I can continue running apps that work well in jails there and use FreeNAS virtualization for the apps that don't.

I do like the simplicity of having everything on one server. I hate the fact every time I want to use a new app that doesn't have a FreeBSD pkg I have to figure out how to build from dependencies and half the time it just can't be done. Meanwhile I know I could have it running in a few minutes or less with Docker. I feel like we are the minority being FreeNAS users, no one seems to care about if their app can work on FreeBSD unless it's a FreeBSD based app and the trend is growing.

 

[danb35]

I feel like we are the minority being freenas users,

Well, that much is true--Linux is much more popular, for better or for worse, than FreeBSD. But an awful lot of apps run just fine on FreeBSD.

 

[joeschmuck]

@mow4cash
How you use your machine should dictate the software you run. As @Chris Moore and @danb35 has mentioned, ESXi is another possiblility, I've been using it for quite some time myself as do others. It's fairly simple to setup however you must understand dependancies if you start to intermingle the programs you install. I also like to use jails where I feel they are best suited, for example I use the Plex program in a jail. I could create a VM and install an OS and then install Plex but that is just too much overhead when I have a simple solution that minimizes my overhead.

If you really want to run multiple operating systems then use something like ESXi, but do you homework on it and read up how to use it. If you have a Windoze computer then you can install VMWare Workstation Player and create VMs and have a blast leanring the basics of ESXi.

 

[toadman]

If you really want to run multiple operating systems then use something like ESXi, but do you homework on it and read up how to use it. If you have a Windoze computer then you can install VMWare Workstation Player and create VMs and have a blast leanring the basics of ESXi.

In that situation one could run a virtual ESXi instance on Player and really play around with ESXi before installing to bare metal.

 

[joeschmuck]

In that situation one could run a virtual ESXi instance on Player and really play around with ESXi before installing to bare metal.

I suspect that you could although I've never tried that.

 

[adrianwi]

I am using ESXi to run all my virtual machines since FreeNAS took away PHP VirtualBox. The only thing that I still use FreeNAS for is the Plex Plug-in Jail and storage.
If the new virtualization platform becomes stable under FreeNAS, I am willing to give it a try. I just couldn't wait for it.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

How easy was it to get ESXi up and running? I'm not planning anything yet, but starting to evaluate things for my next FreeNAS machine and this seems like one of the better options, although it was always frowned upon when I started using FreeNAS some years back.

I'm still running 9.10.1-U4 as the last version to support VirtualBox, although don't currently have any running as I've managed to use bhyve/iohyve to create a ubuntu VM running CrashPlan and docker for some things. Whilst it seems to work reasonably well, it's using much more resources than the previous configuration running 2 VirtualBox VMs. No point hanging onto VirtualBox though as FreeBSD 10.3 will be EOL shortly so I'll need to think about updating. Having played with docker though I'm thinking that's probably the best way to run most of the things I currently use jails for, so think it might be time to just use FreeNAS for the storage and virtualise everything else outside that.

I suspect by the end of the year I'll be pushing the 80% limit, and after 3+ years it's probably better to upgrade and retire my current build to backup duties than try to add more disks. I can also get a motherboard that supports 64GB RAM and get more storage with less drives using 8TB ones.

I'll probably wait to see what happens with 11.2 before making any decisions, but ESXi will certainly be in the mix.

 

[Patrick M. Hausen]

Bhyve (VMs) runs perfectly stable for me on my home server. We run literally hundreds of iocage jails in production in our data centre. On stock FreeBSD, of course.

My experience with Corral and FreeNAS 11 is that most quirks were/are with the UI and automation features, never with the underlying OS technology. It definitely helps a lot if you know your way around FreeBSD without a GUI. If that is the case I don't see any reason not to run iocage jails on FreeNAS. Eventually the GUI will catch up, I'm positive ;)

I'd definitely use iocage with FreeNAS over the GUI warden jails anytime ...

Patrick

 

[Patrick M. Hausen]

Lots of people went all googly-eyed over Docker support in The Release That Must Not Be Named. No doubt there are great things about Docker, but I haven't seen them yet. But the requirement that they run in a Linux VM adds both overhead and resource limitations, neither of which seem like a good thing. And it also makes it more complicated to expose storage to those apps, which is usually needed.

Rumor came to me that initially the lead of Corral development was placing a bet on native Docker support on FreeBSD. When that did not come to pass the VM based solution was chosen as a last minute replacement.

I don't see what's so insanely great [tm] about Docker either, that's why our hosting runs in jails.

Patrick

 

[toadman]

I suspect that you could although I've never tried that.

I've done it on Workstation vs Player, but I'm sure it would work on Player. Some folks do it so they can run multiple nested instances to test things like VSAN where you need 3+ nodes, etc.

 

[Patrick M. Hausen]

I'm still running 9.10.1-U4 as the last version to support VirtualBox, although don't currently have any running as I've managed to use bhyve/iohyve to create a ubuntu VM running CrashPlan and docker for some things. Whilst it seems to work reasonably well, it's using much more resources than the previous configuration running 2 VirtualBox VMs. No point hanging onto VirtualBox though as FreeBSD 10.3 will be EOL shortly so I'll need to think about updating.

This all "just" plain FreeBSD, folks (no insult to the amazing guys at iXsystems intended, honestly). What keeps you from running Virtualbox installed via ports or packages in a plain iocage jail?

E.g. updating is a piece of cake:

Code:

iocage update <jail>
iocage stop <jail>
iocage start <jail>
iocage console <jail>
pkg update
pkg upgrade

A handful of simple commands, what do you need a GUI for?

Patrick

 

[Chris Moore]

Windoze

He said doze... lol

How easy was it to get ESXi up and running?

I sometimes hate to answer questions because I have to say, I messed it up the first time and had to do it over, so I spent a weekend getting it to the point where I could install VMs.

If you want to read about a very well documented build, check this out:
Build Report: Node 304 + X10SDV-TLN4F [ESXi/FreeNAS AIO]
https://forums.freenas.org/index.ph...node-304-x10sdv-tln4f-esxi-freenas-aio.57116/

 

[Nick2253]

My philosophy has always been to segregate storage, so the only services I have running out of jails are related directly to storage: Nextcloud and backup. Otherwise, everything runs in my hypervisor. I use ESXi at work which is excellent for our use-case, but I prefer open source things at home since the free support is usually a lot better. To that end, I currently use XenServer for my hypervisor, but am currently going through the process to migrate to oVirt (KVM meeting CentOS - just my cup of tea!).

If you're interested in setting up a hypervisor (which I recommend if you're serious about your home network :p), I would suggest looking at the KVM-based options, like Proxmox (KVM on Debian) or oVirt. KVM is very popular right now in the cloud space, and therefore the homelab world. And the other major hypervisor player is of course HyperV, if you want to stay in the Microsoft ecosystem.

 

[Patrick M. Hausen]

@Nick2253 ... errrr ... Bhyve is a hypervisor. And I am pretty darn serious about my home network. Just ordered a fresh Supermicro 8-core Atom based server with 64 GB of RAM to have more breathing room for VMs.

Patrick