Example Container-like Reproduceable Jails

[yottabit]

Hi Everyone,

I'm an early adopter because I like to tinker with shiny new things. Every once in a while, this ends up not working out, as was the case with the Corral release. But hey, kudos to the team for making the right decision, and I learned some things in the process.

When I converted to Corral, I lost my Jail configurations. Docker was the new shiny, so fine, I buckled down and decided to learn it. I didn't like the virtualization overhead, or the limit on number of CPUs per container (or maybe VM) of 16, but this was the first time I've been exposed to containerized computing and I did enjoy learning the advantages.

And then Corral was discontinued, so I reverted to 9.10.2. Unfortunately, as an early adopter, I had already converted all of my Jails to Docker containers, and deleted the original Jail datasets. Then I had Docker containers and datasets, but no support for Docker. Soooo... time to create the Jails again.

I applied some of the Docker methodology to my new Jails, and thought some of my friends and coworkers who had done the same upgrade, and then were ready for the same downgrade, would benefit from a write-up. So I did that in a Google Doc, and it kept growing, and eventually a coworker asked that I publish it in GitHub, so I did that, too.

I split each of the configuration scenarios into a separate GitHub project, for better or worse, but it's all available. Hopefully this can help some others with the transition, or employ an easier methodology for the future, or at least give some ideas to run with, as it did my friends and coworkers. Feel free to contribute, too! ;)

And even if you aren't interested at all in Jails or Containers, the scripts I put together showcase some example configurations for Resilio Sync, Google Drive pulls (used to backup my Google Photos, in my case), Google Cloud Storage backup (super cheap with Coldline Storage buckets), Plex Media Server, semi-parallelization of media transcoding, and of course examples of the venerable 1979 Bourne Shell. :D

Google Doc

GitHub Repo

Enjoy!

 

[scrappy]

Nice! Thanks for documenting all this. I use a few jails in my FreeBSD VPS and VMs using iocage. I was a bit disappointed to learn Corral didn't offer jails when it released (not that it matters now) At least we will see the return of jails in FN11 and once I upgrade I will take a closer look at your work as I start playing around with jails again in FreeNAS.

 

[yottabit]

You're welcome!

And for anyone who had problems opening the Google Doc, it appears I had accidentally limited it to be read from my own domain only at some point, and I've now fixed it.

 

[colmconn]

For your parallel transcoding you may want to check out GNU parallel available as the sysutils/parallel package. It will execute N jobs in parallel from a list of jobs of arbitrary length.

(Note that I've not tried the code below to make sure it executes properly.)

Code:

#!/bin/sh

IFS="$(printf '\n\t')"

cat /dev/null > taskfile

while [ $# -gt 0 ]
do

	for name in `find "./$1" \
\( \
-name "*.avi" \
-or -name "*.mkv" \
-or -name "*.mov" \
-or -name "*.mp4" \
-or -name "*.mpg" \
-or -name "*.ts" \
\) \
-and -not -iname "*x265*" \
-type f -print`

	do
	echo "time ffmpeg -loglevel info -y -threads 0 -i "$name" \
		   -vcodec libx265 -strict 2 -threads 0 -preset medium -crf 18 -x265-params level=4.1 \
		   -acodec copy \
		   \"$name.x265.mkv\" && \
		  rm \"$name\"" >> taskfile 
	done &

	# next dir loop
	shift
done

## 16 is the number of jobs to run in parallel
parallel --wd $( pwd ) -j16 < taskFile