SOLVED "Can't Get Shared Memory" inside jail?

[averyfreeman]

Hello,

I'm still pretty new to FreeNAS, but I have a tiny bit of experience with FreeBSD.

I'm running FreeNAS 9.3 on a server I hope to use for hosting to the outside world within a jail.

I'm hoping to use No-IP as my DDNS service. I have it all set up and have used it on past occasions.

The last time I had it working was when I used FreeNAS 9.2. I seem to remember that I set up the tmpfs variable in /etc/fstab within the jail, so I tried to do it again by adding the line:

Code:

tmpfs /tmp tmpfs rw,mode=1777 0 0

However, I am still having the same problem. Invoking

Code:

# df -h

shows that I only have the root file system, and hence, no temporary shared storage system.

I'm sure this is a pretty easy solution for someone who's done it before, but I can't seem to find a definitive answer when searching the threads.

Please, if someone could shed some light on the subject, I'd surely appreciate it!

Thank you,
Avery

 

[averyfreeman]

Note: I found a little bit more information by doing a little experimentation --

I added the line to my /etc/fstab of my FreeNAS distro, mounted the /tmp device using

Code:

# mount -o /tmp

and got the following result when invoking

Code:

# df -h | grep tmp

[root@n00bs3rv3r] /etc# df -h | grep tmp

tmpfs 32M 5.4M 26M 17% /etc

tmpfs 4.0M 8.0k 4M 0% /mnt

tmpfs 2.7G 38M 2.6G 1% /var

So it appears that it's not a limitation of FreeNAS, but rather, a limitation of the jail -

is there a way to configure the jail from the host OS so that it shares the tmpfs with jails?

Please help, this is killing me ...

 

[averyfreeman]

[FACEPALM]I guess I should have looked a little harder (or read the manual ---)[/FACEPALM]

Why not just use Dynamic DNS from services??? It supports no-ip!!!

Read the thread here

This does raise an interesting question for other applications which use shared memory, though --- is there a way to support them from within a jail in FreeNAS 9.3? I could have sworn I got it working in 9.2 ... because I wasn't able to get the DDNS service to work from the GUI, so I remember that I just installed the no-ip client from the ports tree inside the jail...

 

[cyberjock]

Woosh! You went over my head on this one. I am interested to see how/if you can get it working. I'll be watching this thread!

 

[averyfreeman]

Wow, CyberJock - Given your poignant, and most helpful, response to my last thread, it was probably you I was most counting on for steering me in the right direction!

I was thinking of asking the FreeBSD help forum about this one, but I'm pretty sure it's an issue specific to FreeNAS, as I'm sure getting tmpfs to work in FreeBSD is kind of a no-brainer... (include fstab for /tmp device, mount or reboot, voila).

If this is stumping a Forum Guard, could it perhaps be that "New to FreeNAS" is the wrong discussion?

Thanks, anyone who's listening ...

 

[cyberjock]

I've never wanted to (or tried) to share a file system between a jail and the base OS except via the mount functions already included in the WebGUI. I'm actually trying to decide if I think this is a good idea, a fantastic idea, or the dumbest idea I'll see all day. It definitely has some opportunity to go very badly due to edge cases and such.

The best way to share common storage would be mountpoints via the WebGUI... http://doc.freenas.org/9.3/freenas_jails.html#add-storage

 

[averyfreeman]

Well, yes, of course I've explored the option of using storage from the pool or other disks through the GUI, but AFAIK, tmpfs creates a shared memory device used for application cache. I've seen other threads discussing it's use for PostgresSQL, and the inability to get it running in FreeNAS for this reason. So it's not really storage in the conventional sense.

Less esoterically speaking, via the GUI, it's not available as a device in which to mount to the system.

The other thing is, I think perhaps I wasn't clear enough: I definitely don't want the /tmp fs to be accessible system-wide. I tried to configure it in the jail, and when that didn't work and I couldn't find an acceptable solution, I turned to experimenting with it system-wide, since I remember there was something weird like that I did last time, I just can't remember what.

This server is not hosting my data yet, as it is still undergoing a burn-in period before I trust my data to its pool. I might be a n00b, but I'm not suicidal ;)

Anyone else have any ideas? Someone must have some information about tmpfs if such widely used programs that require it to run on FreeBSD are unable to run without in a FreeNAS jail...

 

[CyrilP]

As far as I know the purpose of tmpfs is to mount a ram dir not to share data.

It's used to avoid too much io on disk (or in disk less systems) or if you need a fast storage backend for transient data.

Don't forget that more the space you will use less the ram will be available for your system.

Regarding the mount in jails it's written on the man page.

https://www.freebsd.org/cgi/man.cgi?query=jail&sektion=8

Jails and File Systems
It is not possible to mount(8) or umount(8) any file system inside a jail
unless the file system is marked jail-friendly, the jail's allow.mount
parameter is set, and the jail's enforce_statfs parameter is lower than
2.

Multiple jails sharing the same file system can influence each other.
For example, a user in one jail can fill the file system, leaving no
space for processes in the other jail. Trying to use quota(1) to prevent
this will not work either, as the file system quotas are not aware of
jails but only look at the user and group IDs. This means the same user
ID in two jails share a single file system quota. One would need to use
one file system per jail to make this work.

allow.mount.tmpfs
privileged users inside the jail will be able to mount
and unmount the tmpfs file system. This permission is
effective only together with allow.mount and only when
enforce_statfs is set to a value lower than 2.

enforce_statfs
This determines what information processes in a jail are able to
get about mount points. It affects the behaviour of the follow-
ing syscalls: statfs(2), fstatfs(2), getfsstat(2), and
fhstatfs(2) (as well as similar compatibility syscalls). When
set to 0, all mount points are available without any restric-
tions. When set to 1, only mount points below the jail's chroot
directory are visible. In addition to that, the path to the
jail's chroot directory is removed from the front of their path-
names. When set to 2 (default), above syscalls can operate only
on a mount-point where the jail's chroot directory is located.

You can then apply wanted sysctls by editing your jail from UI.




only available in “Advanced Mode”; comma-delimited list of sysctls to set inside jail (e.g. allow.sysvipc=1,allow.raw_sockets=1)

 

[averyfreeman]

Hi Cyril,

That is definitely some valuable insight you provided, I really appreciate it!

It's used to avoid too much io on disk (or in disk less systems) or if you need a fast storage backend for transient data.

Yes, that makes sense. So, since it is "temorary" storage, that would suggest that it is used for transient storage, right?

Don't forget that more the space you will use less the ram will be available for your system.

Yes, I was concerned about this - I did read a page explaining how to set its limits, but I will have to dig around to figure out where I left it... will post a link to it once I try this out and gain more insight.

The portion of the FreeBSD Jail manpage explains exactly why I wasn't able to do it from within the jail:

JailsandFileSystems

It is not possible to mount(8) or umount(8) any file system inside a jail
unless the file system is marked jail-friendly, the jail's allow.mount
parameter is set, and the jail's enforce_statfs parameter is lower than
2.

I will see about setting the sysctls later on today or tomorrow and post back my results, just in case anyone else is interested in this and wants to hear about it in layman's terms.

The positive thing about this is, even though the tmpfs is created/mounted on the primary file system, the sysctl parameters will allow for flexibility/security regarding access of the fs on an individual basis. That is great news. And, also nice to know that it is not enabled by default in 9.3 (which, I'm guessing it WAS enabled by default in the 9.2 release I tried, since I was able to get it to work without knowing any of this stuff).

only available in “Advanced Mode”; comma-delimited list of sysctls to set inside jail (e.g.allow.sysvipc=1,allow.raw_sockets=1)

Thanks for mentioning that as well, as it probably saved another post of me asking, "Where's the button?!"

Awesome! Can't wait to try it out! Report back soon...

 

[CyrilP]

Well I gave a try an unfortunately allow.mount.tmpfs is not yet available (came with FreeBSD 10).

https://www.freebsd.org/cgi/man.cgi?query=jail&apropos=0&sektion=8&manpath=FreeBSD 9.3-stable&arch=default&format=html

No mention of tmpfs on this one.

What you can do but it's pretty ugly is to.

On freenas host create 10KB tmpfs:

Code:

[root@freenas] ~# mount -t tmpfs -o size=10000 tmpfs /bla

Then use nullfs (UI) to mount this /bla to a /test on your jail.

From your jail and with the proper sysctl options above, you will see:

Code:

# mount
test/jails/instance_1 on / (zfs, local, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
procfs on /proc (procfs, local)
/bla on /test (nullfs, local)

And

Code:

# dd if=/dev/urandom of=/test/file count=1 bs=512
1+0 records in
1+0 records out
512 bytes transferred in 0.000046 secs (11126858 bytes/sec)
# dd if=/dev/urandom of=/test/file count=1 bs=512
1+0 records in
1+0 records out
512 bytes transferred in 0.000040 secs (12782641 bytes/sec)
# dd if=/dev/urandom of=/test/file count=10bs=512
10+0 records in
10+0 records out
5120 bytes transferred in 0.000825 secs (6204807 bytes/sec)
# dd if=/dev/urandom of=/test/file count=2 bs=512
dd: /test/file: No space left on device
17+0 records in
16+0 records out
8192 bytes transferred in 0.000430 secs (19046418 bytes/sec)
# df -ha
Filesystem               Size    Used   Avail Capacity  Mounted on
test/jails/instance_1    7.2G    2.1G    5.0G    30%    /
devfs                    1.0k    1.0k      0B   100%    /dev
procfs                   4.0k    4.0k      0B   100%    /proc
/bla                      12k     12k      0B   100%    /test

But, because there is a but... deleting files will not free up space on your tmpfs mountpoint exposed using nullfs... due to inode caching. You may try with a larger tmpfs volume but I'm not sure if it will finally free up inodes.

 

[averyfreeman]

Crap, I don't even know what an inode is yet. I feel pretty behind.

I realized this morning why I was thinking it was shared memory - I was reading a thread about the shm counterpart in FreeBSD. shm is the shared memory device mounted in fstab for Linux. The thread discussed tmpfs in FreeBSD as being essentially synonymous to shm in Linux.

The No-IP client I compiled in The FreeNAS jail quit while returning the complaint "Can't Get Shared Memory", which is the reason for the name of the thread. Do you think it's just a carry-over from the client originally being written in Linux and ported to FreeBSD?

 

[CyrilP]

Oh ok, this as nothing to deal with tmpfs or shm but about IPC

Try this settings in your jail:

allow.sysvipc
A process within the jail has access to System V IPC
primitives. In the current jail implementation, System V
primitives share a single namespace across the host and
jail environments, meaning that processes within a jail
would be able to communicate with (and potentially inter-
fere with) processes outside of the jail, and in other
jails.