FreeNAS 11 + IOCAGE. How Do I Mount Host Datasets Inside Jails

[scrappy]

Now that iocage is functional in FreeNAS 11-RC4 I have created a few jails to play with. Today I decided to test out the Crashplan iocage plugin. No problem getting the Crashplan jail plugin up and running. This brings me to the real question: How do I mount FreeNAS host datasets inside the jail? I am feeling a bit confused how I go about using iocage zfs set options.

 

[StarkJohan]

I'm having partial success trying the following. I'm trying to mount a dataset that is in another pool but that seems harder. Keep in mind that this is just me trying stuff from the man page. https://www.freebsd.org/cgi/man.cgi?query=iocage&sektion=8

With jail stopped:

Code:

sudo iocage set jail_zfs=on <jailname> 
sudo iocage set jail_zfs_dataset=<dataset> <jailname>

Second line is optional, not using it will give you option to mount other jailed datasets IIRC.
After starting jail, from within the jail you should be able to see available zfs stuff and then mounting using:

Code:

zfs list
zfs set mountpoint=/mypath <dataset>

I'd like to get the jail ip using DHCP but I haven't figured that out yet. Seems like these options should get me in the right direction at least.

Code:

vnet=on dhcp=on bpf=on

 

[StarkJohan]

Mounting the dataset from within the jail seems to change the dataset mount point in the freeness system, this can't be right.

I'm also getting some python error on starting and stopping the jails if I have jail_zfs=on.

Code:

% sudo iocage stop testjail

Password:

* Stopping 574f4415-948c-4dde-a73f-9181d9d0bfe4 (testjail)

  + Running prestop OK

  + Stopping services OK

Traceback (most recent call last):

  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_stop.py", line 143, in __stop_jail__

	child], stderr=su.STDOUT)

  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_common.py", line 421, in checkoutput

	out = su.check_output(*args, **kwargs)

  File "/usr/local/lib/python3.6/subprocess.py", line 336, in check_output

	**kwargs).stdout

  File "/usr/local/lib/python3.6/subprocess.py", line 418, in run

	output=stdout, stderr=stderr)

subprocess.CalledProcessError: Command '['setfib', '0', 'jexec', 'ioc-574f4415-948c-4dde-a73f-9181d9d0bfe4', 'zfs', 'umount', 'ssd-mirror-pool/testjail']' returned non-zero exit status 1.


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "/usr/local/bin/iocage", line 10, in <module>

	sys.exit(cli())

  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__

	return self.main(*args, **kwargs)

  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main

	rv = self.invoke(ctx)

  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke

	return _process_result(sub_ctx.command.invoke(sub_ctx))

  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke

	return ctx.invoke(self.callback, **ctx.params)

  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke

	return callback(*args, **kwargs)

  File "/usr/local/lib/python3.6/site-packages/iocage/cli/stop.py", line 32, in cli

	ioc.IOCage(jail, rc=rc).stop()

  File "/usr/local/lib/python3.6/site-packages/iocage/lib/iocage.py", line 500, in stop

	ioc_stop.IOCStop(uuid, tag, path, conf, silent=self.silent)

  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_stop.py", line 27, in __init__

	self.__stop_jail__()

  File "/usr/local/lib/python3.6/site-packages/iocage/lib/ioc_stop.py", line 157, in __stop_jail__

	err.output.decode("utf-8").rstrip()))

RuntimeError: cannot unmount 'ssd-mirror-pool/testjail': not currently mounted

 

[scrappy]

This is very similar to the problem I am having. After issuing the command: jail_zfs=on <jailname> all I could see from inside my jail is the zpool that iocage has set as its default. This might be Ok for most, but I have three zpools on my FreeNAS server which the jail needs access to, but cannot see using zfs list. Something is amiss here.

Also, after I tried jail_zfs_dataset it seems to have moved the jail's root path and nothing I do changes it back.

 

[StarkJohan]

Just to clarify.

If I'm creating a fresh jail and have a dataset in the same pool named "test", accessing it works if I'm not changing the mountpoint from within the jail. Changing mountpoint really seems to screw things up or I'm doing it totally wrong...

This works:

Code:

sudo iocage set jail_zfs=on <jailname>
sudo iocage set jail_zfs_dataset=test <jailname>

But what then happens is that the mountpoint also changes on the main system?! Before carrying out the above commands a sample file in the dataset "test" in the pool "ssd-mirror-pool" called test.txt is accessible from the main system at this location:

Code:

/mnt/ssd-mirror-pool/test/test.txt

After setting upp access from within the jail "testjail" the dataset disappears from the main system in the previously mentioned location but is now instead accessible "in the jail":

Code:

/mnt/iocage/jails/325dbc0f-52ed-424d-b08b-4710f50b19ee/root/mnt/ssd-mirror-pool/test/test.txt

This can't be right, right?

 

[StarkJohan]

Here's another approach that might be worth checking out. Haven't tested it myself yet.

http://pid1.com/posts/post24.html

Edit: Yeah, mounting nullfs "straight into the jail" works like a charm. However, I have not idea if this is advisable in any way. I would love some input from "the experts".

Code:

mount -t nullfs -o ro /mnt/<some pool>/<dataset>/<dir> /mnt/iocage/jails/<jail ID>/root/<mountpoint>

 

[scrappy]

Just to clarify.

If I'm creating a fresh jail and have a dataset in the same pool named "test", accessing it works if I'm not changing the mountpoint from within the jail. Changing mountpoint really seems to screw things up or I'm doing it totally wrong...

This works:

Code:

sudo iocage set jail_zfs=on <jailname>
sudo iocage set jail_zfs_dataset=test <jailname>

But what then happens is that the mountpoint also changes on the main system?! Before carrying out the above commands a sample file in the dataset "test" in the pool "ssd-mirror-pool" called test.txt is accessible from the main system at this location:

Code:

/mnt/ssd-mirror-pool/test/test.txt

After setting upp access from within the jail "testjail" the dataset disappears from the main system in the previously mentioned location but is now instead accessible "in the jail":

Code:

/mnt/iocage/jails/325dbc0f-52ed-424d-b08b-4710f50b19ee/root/mnt/ssd-mirror-pool/test/test.txt

This can't be right, right?

Yes. Exactly what happens to me.

 

[scrappy]

Here's another approach that might be worth checking out. Haven't tested it myself yet.

http://pid1.com/posts/post24.html

Edit: Yeah, mounting nullfs "straight into the jail" works like a charm. However, I have not idea if this is advisable in any way. I would love some input from "the experts".

Code:

mount -t nullfs -o ro /mnt/<some pool>/<dataset>/<dir> /mnt/iocage/jails/<jail ID>/root/<mountpoint>

nullfs is what I have done in the past on FreeBSD and is a perfectly reasonable method to mount host directories in a jail. I just thought iocage had its own method to mount zfs datasets inside the jail somehow. Maybe I (we) have mistaken the correct usage of jails_zfs_dataset=<foo> <jail> unless it is broken under FreeNAS. All I know right now is I cannot even use iocage at the moment (probably until I reboot, or figure out how/where to clear iocage's temp files) because it currently expects there to be jails where it can no longer find them.

 

[StarkJohan]

nullfs is what I have done in the past on FreeBSD and is a perfectly reasonable method to mount host directories in a jail. I just thought iocage had its own method to mount zfs datasets inside the jail somehow.

It would be interesting to know what method is to be used when setting up jail storage in the GUI in 11.1.

All I know right now is I cannot even use iocage at the moment (probably until I reboot, or figure out how/where to clear iocage's temp files) because it currently expects there to be jails where it can no longer find them.

I ran into the same problem one time and another time the jails couldn't be stopped at all. I had to remove the iocage dataset. Just activated and ran fetch again and stuff was back to normal.

 

[Brandon Schneider]

Lots to reply to here, so I'm going to just give the cliff notes and keep it brief ;)

1 - Jailed datasets are *exclusive* to the jail. So changing their mountpoint inside the jail is going to reflect in zfs list. That is how the process works.

2 - DHCP isn't ported to the rewrite yet, that's a Soon™ feature. The docs you linked to from FreeBSD's site are for the old version, and the iocage.readthedocs.io are also for the legacy implementation. Only so much time in a day ;) So in the meantime use the manpage included in FreeNAS (partially out of date, I know I suck at documenting :() or use

Code:

iocage COMMAND --help

3 - NullFS is definitely how you want to approach this, and good news! I have your back. iocage includes an easy command for this,

Code:

iocage fstab

and in the case of adding a nullfs mount, just issue

Code:

iocage fstab -a JAIL /the/directory/to/share

4 - iocage has no temporary files. Changing the mountpoint of

Code:

POOL/iocage

is fine, and if iocage is having any trouble seeing them, I suggest trying

Code:

zfs mount -a

and trying again. But to avoid unexpected headaches, I'd leave it at default. Default is good, default is your friend.

I hope this clears up some stuff guys, thanks for trying iocage out! If you want a more immediate answer, use the github at https://github.com/iocage/iocage, watching the forums is not my strength!

 

[scrappy]

Lots to reply to here, so I'm going to just give the cliff notes and keep it brief ;)

1 - Jailed datasets are *exclusive* to the jail. So changing their mountpoint inside the jail is going to reflect in zfs list. That is how the process works.

2 - DHCP isn't ported to the rewrite yet, that's a Soon™ feature. The docs you linked to from FreeBSD's site are for the old version, and the iocage.readthedocs.io are also for the legacy implementation. Only so much time in a day ;) So in the meantime use the manpage included in FreeNAS (partially out of date, I know I suck at documenting :() or use

Code:

iocage COMMAND --help

3 - NullFS is definitely how you want to approach this, and good news! I have your back. iocage includes an easy command for this,

Code:

iocage fstab

and in the case of adding a nullfs mount, just issue

Code:

iocage fstab -a JAIL /the/directory/to/share

4 - iocage has no temporary files. Changing the mountpoint of

Code:

POOL/iocage

is fine, and if iocage is having any trouble seeing them, I suggest trying

Code:

zfs mount -a

and trying again. But to avoid unexpected headaches, I'd leave it at default. Default is good, default is your friend.

I hope this clears up some stuff guys, thanks for trying iocage out! If you want a more immediate answer, use the github at https://github.com/iocage/iocage, watching the forums is not my strength!

That was exactly what I needed to know! Thank you so much!

 

[StarkJohan]

If I setup my jails using iocage on 11-RC4, will I be able to manage them from the GUI once 11.1 is released or will I have to rebuild them again?
My old jails are all on the 9.3 base and needs to be migrated but I'd rather hold off a few more months if it means I don't have to do the work twice...

 

[Brandon Schneider]

If I setup my jails using iocage on 11-RC4, will I be able to manage them from the GUI once 11.1 is released or will I have to rebuild them again?
My old jails are all on the 9.3 base and needs to be migrated but I'd rather hold off a few more months if it means I don't have to do the work twice...

You should be good to go creating them now from the CLI with iocage and using them in 11.1 when iocage makes the GUI debut. Anything created in the GUI currently still uses Warden, but come 11.1 that should change.

 

[StarkJohan]

You should be good to go creating them now from the CLI with iocage and using them in 11.1 when iocage makes the GUI debut.

Thanks for confirming this!

 

[adrianwi]

Slightly off topic, but how does iocage deal with future upgrades to FreeBSD?

At the minute, jails created in a previous version of FreeNAS (lets say 9.3.1) don't allow for updates/upgrades when the FreeNAS system has been updated to a newer version of FreeBSD (say 9.10 or 11.x)

Will jails created using iocage in FreeNAS 11 (FreeBSD 11) be fully functional when FreeNAS is updated to FreeBSD 12, or is this just a jail issue regardless of creating using warden or iocage?

 

[StarkJohan]

Slightly off topic, but how does iocage deal with future upgrades to FreeBSD?

My understanding is that the "iocage upgrade" and "iocage update" takes care of this and updating jails should be possible but I have no first hand experience of this as I'm just getting started with iocage. Keep in mind that I might be misunderstanding these commands totally...

 

[pschatz100]

3 - NullFS is definitely how you want to approach this, and good news! I have your back. iocage includes an easy command for this,

Code:

iocage fstab

and in the case of adding a nullfs mount, just issue

Code:

iocage fstab -a JAIL /the/directory/to/share

Plex users: be mindful that, after you add storage to your new Plex jail, Plex may not allow you to simply "browse" to the data - you may have to type in the full path to the data. This was the case for me as my jails and data are on different volumes.

 

[StarkJohan]

Plex users: be mindful that, after you add storage to your new Plex jail, Plex may not allow you to simply "browse" to the data - you may have to type in the full path to the data. This was the case for me as my jails and data are on different volumes.

Same problem here, manual entry works fine.

 

[booya]

Hi all!

Perhaps a bit OT but I'm trying to read up on jails in general and I had my first iocage experience yesterday. I've got three questions for you:

1. My FreeNAS host (11.0-U1) has two interfaces and I would like to assign the secondary interface to the jail, is this possible? It seems that the jail still wants to use part of the hosts primary connection (eg. defaultrouter in the jail is that of the hosts primary interface).

2. Is it possible to mount a read-write dataset in the jail while at the same time sharing it through FreeNAS/SMB (read privileges only)?

3. Got the: "Error occured, 11.0-RELEASE was not updated to the latest patch level.". Is it unsafe to let the jail out on the interwebz? :)

 

[Teeps]

Thank you so much for this thread. I think this is the only place on the whole internet for this info ;)