NFSv4 Home Directories with autofs ACL Help

[gwaitsi]

i have linux clients using autofs, with home directory mapping based on uid/gid.
additionally i have shared media directories and for me only working directories.

these were working when i was on OMV.
i created the same users with the same uid/gid as i had in OMV and freenas has the old IP address.

data (pool)
- home (set) -> user1 (set)
- home (set) -> user2 (set)
- media (set)
- working (set)

media & working mappings are working (so NFS per say is good), but the home mapping are not.

a device for home is created e.g. /media/home/userx
and does that correctly for each user that logs in, but there's content underneath

obviously i haven't got the ACLs correct.
current ACLs are;
- home (set) root:users ;
owner@ Allow / Advanced / All / Basic / No Inherit
group@ Allow / Basic / Traverse / Basic / Inherit

- user1(set) user1:user1;
owner@ Allow / Advanced / all / Basic / Inherit
group@ Allow / Advanced / all / Basic / Inherit

what i am doing wrong?
i guess it is because the nfs client can't read the data(pool) / home (set) ?

 

[KrisBee]

By "sets", I assume you mean "datasets", but what is "device" in this context, is it just a directory? And what is this other visible content?

IIRC, OMV creats a NFSv4 export for NFS shares, but linux clients can mount this as NFSv3 or NFSv4. If you never modified the NFS shared folder's ACL in OMV, and are relying on standard uid/gid perms for share access, then FreeNAS NFSv4 ACLs may not be relevant to you. If you have made changes within the FreeNAS ACL editor, you may have added ACLs to your FreeNAS datasets that you do not need. I would start with std unix perms on the datasets you wish to share by NFS and ensure those datasets were created with "share type = generic" (FreeNAS guide 10.2.10)

If you've created one child dataset per user under a home dataset, then you will need a NFS share for each of those child datasets.

Then you will have to check the autofs configs on your linux clients, checking the contents of whatever aufo map files you've added to auto.master and any mount paths referenced in those map files and fstype.

 

[gwaitsi]

i clearly have a misunderstanding with something.
my media dataset was accessible via NFS, but plex could not read it.

then i re-added user 972 and plex worked, but now the nfs is not accessible
(although i can see the folder on the client, but when clicking says it can not be found)

owner = me(uid 1000):users(gid 100)
ACLs
owner@ = Type:Allow, Permission Type:Basic, Permissions: Full, Flags:Basic, Flags: Inherit
group@ = Type:Allow, Permission Type:Basic, Permissions: Read, Flags:Basic, Flags: Inherit
User:972 = Type:Allow, Permission Type:Basic, Permissions: Full, Flags:Basic, Flags: Inherit

on the client
auto.nfs
media -fstype=nfs4,rw,auto,timeo=14,intr,iocharset=utf8,uid=$USER,gid=users nas:/mnt/data_volume/nas_media/

auto.home
me -fstype=nfs4,rw,auto,timeo=14,intr,iocharset=utf8,uid=$USER,gid=users nas:/mnt/data_volume/nas_home/&

as i say,
the client worked under omv, i just had to update the path and the auto.nfs worked after updating the path....
but not in plex, until i added 972 which fixed plex, but broke the nfs.

haven't been able to get the home working at all

------ another example -------
photos
owner = me(uid 1000):users(gid 100)
ACLs
owner@ = ACL type: Allow, Type:Advanced, Permission Type:All, Permissions: all except delete selected, Flags:Basic, Flags: Inherit
group@ = ACL type: Allow, Type:Advanced , Permission Type:Basic, Permissions: Read, Flags:Basic, Flags: Inherit

photos / user1
owner = me(uid 1000):users(gid 100)
ACLs
owner@ = ACL type: Allow, Type:Advanced, Permission Type:All, Permissions: all except delete selected, Flags:Basic, Flags: Inherit
group@ = ACL type: Allow, Type:Advanced , Permission Type:Basic, Permissions: Read, Flags:Basic, Flags: Inherit

** tried group@ and group = users

photos / shared
owner = me(uid 1000):users(gid 100)
ACLs
owner@ = ACL type: Allow, Type:Advanced, Permission Type:All, Permissions: all except delete selected, Flags:Basic, Flags: Inherit
group@ = ACL type: Allow, Type:Advanced , Permission Type:All, Permissions: Read, Flags:Basic, Flags: Inherit

can see in the mounts, but no user inc myself can access

 

[KrisBee]

The things you need to sort out:
=================================

1. When you were using OMV, if you were not using (POSIX)ACLs why are you using extended NFSV4 ACLs now with FreeNAS and NFS shares?
2. Can you achieve what you want by using std unix perms? In the case of plex, can you access data as you want by creating a plex user on FreeNAS and granting access to data via a group which has plex and other users as members?
3. Why are you using NFSv4? Simply because that's how OMV and your linux clients were set up? The FreeNAS NFS server can run as a NFS v3 only server.
4. The FreeNAS/FreeBSD manpage for exports says this: "All ZFS file systems in the subtree below the NFSv4 tree root must be exported". As every dataset is a different file system, if you use datasets within datasets just exporting the parent dataset does not make any child dataset acessible. See FreeNAS guide section: 13.3
5. What's the consequence of 4 above for your auto.home. You are using a wildcard &, AFAIU this means "all the directories found in the /home directory on the server will be mapped to a directory of the same name on the client", e.g nas_home/user1, nas_home/user2 etc. So are you exporting all the associated datasets as seperate nfs shares or just nas_home?
6. Do NFSv3 and NFSv4 NFS server shares function differently in FreeNAS/FreeBSD? Is using the "--alldirs" flag with NFSv3 NFS share relevant? See this thread for example and in particular @Fredda's replies: https://www.ixsystems.com/community/threads/sharing-nfs-smb-is-going-to-kill-me.85586/
7. In the case of mounting remote home shares via NFS, what are the pros and cons of making separate datasets per user under a parent dataset, as opposed to separate directories under a single parent dataset?

I'd also add that AFAIU the ACL editor introduced in recent versions of FreeNAS was primarily intended for use with SMB shares. Before this users had to edit share ACLs via a Windows client. With care it can be used to edit ACLs for NFSv4 shares. But your are forced to set some form of inheritance on the ACL and I would want to check the results using "getfacl" in the FreeNAS shell, particularly if the "Apply permissons recursively" option is used. Not having an ACE for everyone@ on a dataset can mean accees is denied because the full path cannot be traversed.

Did you re-write your auto. files? You seem to have used options which are for mount.cifs and not mount.nfs in linux, e.g iocharset. Check your linux logs.

 

[gwaitsi]

So i have samba setup and verified all clients operating the way they should, to try and eliminate any ACL issues.

turning back to the shared directories first, i found;

NFS Shares
photos dataset
i had to set maproot users = root and mapall users = --- / groups = ---

photos user dataset
i had to set maproot users = root and mapall users = --- / groups = ---

- photos work as intended in both linux / windows with the original auto.nfs client settings

for the home directories, i tried the same principle and all users could see all other users data.
mapall users/groups = nobody (per the doc recommendation) breaks the mapping completely.

so, home directories is working with

NFS Shares
home dataset
i had to set maproot users = root and mapall users = --- / groups = ---

home user dataset
i had to set maproot users = root and mapall users = --- / groups = ---

In summary, the main issue was not creating additional shares/exports for each individual user under the home dataset share.
this was not required under OMV.

 

[KrisBee]

Didn't get round to mentioning the maproot/mapall settings in FreeNAS nfs shares, but looks like your sorted. Another thing to note is that If your sharing the same data via NFS and SMB at the same time, that can be problematic.

About autofs, it may have been around a long time before systemd automounts, but I've made little use of it. I had to check man 5 autofs page, as your fstype options looked odd to me. To quote:

-fstype=
is used to specify a filesystem type if the filesystem is not of the default NFS type. This option is processed by the automounter and not by the mount command.

So what does automounter do with an option like iocharset for an nfs mount as its an invalid parameter? Stop the autofs service and start autmount in the foreground with "automount -f -debug" and you'll see it makes a nfs mount call with an -s switch. It makes a "sloppy mount" which ignores options not understood by mount.nfs. So you can trim the fat from your auto. map files as a lot of those fstype options are redundant.

 

[gwaitsi]

thanks for the advice. i looked at the manual and come up with these corrected values for auto.nfs
media -fstype=nfs,soft,timeo=14,intr,rsize=8192,wsize=8192 nas:/mnt/data_volume/nas_media/

i did the same below for auto.home
* -fstype=nfs,soft,timeo=14,intr,rsize=8192,wsize=8192 nas:/mnt/data_volume/nas_home/&

but the user names don't seem to be substituted.
if a change the * for a specific user name it works.

 

[KrisBee]

Hmm, your auto.home format looks OK to me for an indirect map. Although I think you can leave the client to negotiate rsize & wsize with the NFS server.

What's the associated line in auto.master? PS What linux distro is this?

 

[gwaitsi]

linux mint 20 & 19.3

i have the following errors in the status

i get the below error whether or not i use user1 or *

automount key ".xdg-volume" not found in map source
automount key "autorun.inf" not found in map source

auto.master =
/media/home /etc/auto.home --timeout=60 --ghost

 

[KrisBee]

All I can say is that's I've just tested indirect map with wildcard on a deb9 install (openbox DE) and it's working without error. After editing your auto. map files you did stop the autofs service and umount any nfs shares before restarting the autofs service.

my auto.master line:

Code:

/mnt/homedirs /etc/auto.fnshare --timeout=60 --ghost

my auto.fnshare map:

Code:

* -fstype=nfs 192.168.0.22:/mnt/NasPool/homedirs/&

The automounts:

Code:

chris@sweep:~$ ssh chris@192.168.0.90
chris@192.168.0.90's password:
Linux helium-vm 4.9.0-13-amd64 #1 SMP Debian 4.9.228-1 (2020-07-05) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Mon Aug 31 18:05:19 2020 from 192.168.0.201
chris@helium-vm:~$ cd /mnt/homedirs/
chris@helium-vm:/mnt/homedirs$ ls -l
total 1
drwxr-xr-x 2 chris chris 4 Aug 29 16:45 chris
chris@helium-vm:/mnt/homedirs$ cd chris
chris@helium-vm:/mnt/homedirs/chris$ ls -l
total 1
-rw-r--r-- 1 chris chris 5 Aug 29 15:30 testdoc
-rw-r--r-- 1 chris chris 0 Aug 29 16:45 testdoc2
chris@helium-vm:/mnt/homedirs/chris$ sudo -i
root@helium-vm:~# su - fred
fred@helium-vm:~$ cd /mnt/homedirs/fred
fred@helium-vm:/mnt/homedirs/fred$ ls -l
total 1
-rw-r--r-- 1 fred fred 0 Aug 31 16:29 freddoc
fred@helium-vm:/mnt/homedirs/fred$ nfsstat -m
/mnt/homedirs/chris from 192.168.0.22:/mnt/NasPool/homedirs/chris
Flags: rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.90,local_lock=none,addr=192.168.0.22

/mnt/homedirs/fred from 192.168.0.22:/mnt/NasPool/homedirs/fred
Flags: rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.90,local_lock=none,addr=192.168.0.22

fred@helium-vm:/mnt/homedirs/fred$ 

I note this bug report: https://bugs.launchpad.net/ubuntu/+source/autofs/+bug/1855318 , otherwise this look one for the Mint forums. Does it still error out if you don't use the --ghost option?

EdIt: Just seen this jounralcltl dump on paste bin: https://pastebin.com/vDDrvz0Q Same (?) error, seems to be a Gnome desktop thing.

 

[gwaitsi]

if i run automount -d -f -v i get

mount(nfs): calling mkdir_path /media/home/user1
mount(nfs): calling mount -t nfs -s -o rw,nosuid,soft nas:/mnt/data/nas_home/user1 /media/home/user1
>> mount.nfs: mounting nas:/mnt/data/nas_home/autorun.inf failed, reason given by server: No such file or directory
mount(nfs): mount failure nas:/mnt/data/nas_home/autorun.inf on /media/home/autorun.inf
dev_ioctl_send_fail: token = 140
failed to mount /media/home/autorun.inf

this file doesn't exist on the server, so i am not sure where it is supposed to come from.

** i am getting close. I created two directories in the nas_home dataset
autorun.inf
.xdg-volume-info

now if i login, i get my home directory mapped - but not any of the other users. i guess it is to do with the rights of those two directories?

 

[KrisBee]

Look again at links to bug reports in my last post. Check your Mint installs don't have any junk left in the dirs you target in automount. Adding random stuff to your FreeNAS to get a working automount makes no sense at all. Running "automount -d -f -v" should give you a clean result. Here's an extract of the output:

Code:

attempting to mount entry /mnt/homedirs/chris
lookup_mount: lookup(file): looking up chris
lookup_mount: lookup(file): chris -> -fstype=nfs 192.168.0.22:/mnt/NasPool/homedirs/&
parse_mount: parse(sun): expanded entry: -fstype=nfs 192.168.0.22:/mnt/NasPool/homedirs/chris
parse_mount: parse(sun): gathered options: fstype=nfs
parse_mount: parse(sun): dequote("192.168.0.22:/mnt/NasPool/homedirs/chris") -> 192.168.0.22:/mnt/NasPool/homedirs/chris
parse_mount: parse(sun): core of entry: options=fstype=nfs, loc=192.168.0.22:/mnt/NasPool/homedirs/chris
sun_mount: parse(sun): mounting root /mnt/homedirs, mountpoint chris, what 192.168.0.22:/mnt/NasPool/homedirs/chris, fstype nfs, options
mount_mount: mount(nfs): root=/mnt/homedirs name=chris what=192.168.0.22:/mnt/NasPool/homedirs/chris, fstype=nfs, options=
mount_mount: mount(nfs): nfs options="", nobind=0, nosymlink=0, ro=0
get_nfs_info: called with host 192.168.0.22(192.168.0.22) proto 6 version 0x30
get_nfs_info: nfs v3 rpc ping time: 0.000000
get_nfs_info: nfs v2 rpc ping time: 0.000000
get_nfs_info: host 192.168.0.22 cost 0 weight 0
get_nfs_info: called with host 192.168.0.22(192.168.0.22) proto 17 version 0x30
prune_host_list: selected subset of hosts that support NFS3 over TCP
mount_mount: mount(nfs): calling mkdir_path /mnt/homedirs/chris
mount_mount: mount(nfs): calling mount -t nfs 192.168.0.22:/mnt/NasPool/homedirs/chris /mnt/homedirs/chris
mount_mount: mount(nfs): mounted 192.168.0.22:/mnt/NasPool/homedirs/chris on /mnt/homedirs/chris
dev_ioctl_send_ready: token = 3
mounted /mnt/homedirs/chris

Which desktop are you running on Mint?

 

[gwaitsi]

i'm using cinnamon, but both of these appear to be gnome related

Bug 551403 – Support Reading Volume Icon/Name Information from .xdg-volume-info

bugzilla.gnome.org

but when i tried to create them as files, it didn't work, but did when i created them as directories.

like i say, my home drive is created along with the autorun.inf directory under /media/home
but for a standard user the autorun.inf directory is created, but not their user directory.

the only difference between myself and the other users from getfacl is the "fd" for the group. they are missing this, otherwise they are the same

 

[KrisBee]

As a test , change your automounts to be under /mnt rather than /media this might avoid problems with your File Manager,/ gvfs, etc. I only run debian/ubuntu, so you'll have to take your problem to the Mint forum if it persists.

 

[gwaitsi]

As a test , change your automounts to be under /mnt rather than /media this might avoid problems with your File Manager,/ gvfs, etc. I only run debian/ubuntu, so you'll have to take your problem to the Mint forum if it persists.

for the non-home directories,
it doesn't matter if they are mounted under mnt or media
- only the desktop icons are not created if mounted under mnt.
- they work for all users

for the home directories mounted under mnt,
- there are no errors, yet it is not mounted for my account. (seems like not mounting at all)

for the home directories mounted under media
- my account works
- it is not mounted for the other accounts but the autorun.inf directory is

i will try to install in a virtual box with xfce or a different distro and see what happens

 

[KrisBee]

This probably doesn't help much, but I found time to download and install latest Cinnamon Mint 20 in a VM. Your error is not present on a fully update install, at least not for a similar config to yours:

line in /etc/auto.master:

Code:

/media/NAS  /etc/auto.fnshare --timeout 60 browse

/etc/auto.fnshare:

Code:

* -fstype=nfs 192.168.0.22:/mnt/NasPool/&

FreeNAS export:

Code:

V4: / -sec=sys
/mnt/NasPool/chris -maproot="root":"wheel" -network 192.168.0.0/24
/mnt/NasPool/fred -maproot="root":"wheel" -network 192.168.0.0/24

FreeNAS datasets - generic with unix perms:

Code:

root@freenas[/mnt/NasPool]# getfacl chris
# file: chris
# owner: chris
# group: chris
            owner@:rwxp--aARWcCos:-------:allow
            group@:r-x---a-R-c--s:-------:allow
         everyone@:------a-R-c--s:-------:allow
root@freenas[/mnt/NasPool]# getfacl fred
# file: fred
# owner: fred
# group: fred
            owner@:rwxp--aARWcCos:-------:allow
            group@:r-x---a-R-c--s:-------:allow
         everyone@:------a-R-c--s:-------:allow
root@freenas[/mnt/NasPool]# 

A user still would have to navigate to /media/NAS/username on Mint before the FreeNAs share is mounted. There's also the fact that action of logging out doesn't automatically unmount their FreeNAS share. So changing users can led to the situation in the screen grab.

 

[gwaitsi]

thanks for the time you are putting in on me. appreciate it.
i tried a couple of distros in vm, and got more or less the same results, as the desktop.

i made sure my users/groups have the same id on freenas / distro
e.g. user1:user1 1000:1000, user2:user2 1001:1001 and user group 100

my permissions seem to be quite different to your own.

Code:

#  file: nas_home (dataset)
# owner: nobody
# group: users
    owner@:rwxpD-aARWcCos:-------:allow
    group@:r-x---a-R-c--s:fd-----:allow
 everyone@:-------------s:fd-----:allow
 
#  file: user1
# owner: user1
# group: users
    owner@:rwxp--aARWcCos:-------:allow
    group@:rwxp--a-R-c--s:-------:allow
 everyone@:r-x---a-R-c--s:-------:allow
 
#  file: user2
# owner: user2
# group: users
    owner@:rwxp--aARWcCos:-------:allow
    group@:rwxp--a-R-c--s:-------:allow
 everyone@:r-x---a-R-c--s:-------:allow
  

 

[KrisBee]

I kept my example as simple as possible to create automounts in the way you seemed to want. I did not use the ACL editor in FreeANS, nor "setfacl" at the FreeNAS CLI. I just used std unix perms. Nor have I used nested datasets. You choose not to answer my questions about why you are using ACLs in #2 & #4 above.

I don't understand what you are trying to achieve with the ACL/ACEs on the dataset "nas_home". Assuming "user1" and "user2" really are child datasets which you exported, then they are their own filesystem with std unix perms. In your case, user1 and user2 on your linux client will have rw access to one another's files if both shares are mounted.

IIRC, OMV uses the SGID bit on all share folders to enable co-operative sharing and users are not given a primary group that matches their uid but are members of the"users" group (gid 100). Perhaps you are re-crating this config on FreeNAS, but your are free to devise alternatives.

 

[gwaitsi]

so i probably messed it up, when i was trying to get my data across with all the import problems

i originally created three datasets
- home
- media
- photos

from omv, i copied in the data under directories
- home
user1
user2
- photos
user1
user2

when i couldn't see the user directories in the gui, i created a data set for each user under home and photos.
thats when i started messing with the rights and changing from root:wheel and trying to replicate what i had in OMV.

if we go back to basics, and fix the way freenas should be setup.
- should i have one dataset for each type only, or each type and user?
- for the top level datasets, i should leave them with the default root:wheel?
- if not a dataset for the users, then the rights and groups have to be set via the shell?

i think it is necessary to fix these basics before tying to solve the shares, then they might fix themselves

also,
- my users are all on windows
- i'm on windows and linux but not usually at the same time (so no file conflicts)
- i have coreelec boxes using cif shares (which so far haven't worked)

 

[gwaitsi]

so, i have removed the user data sets under home and photos and created directories with simply permissions.

for the photo data set, i made me:users the owner with 755 permissions

for the users under photos,
i made ownership user#:users
i tried to create directories with 750 and files with 640 and it didn't work.

when i changed directories to 755 i could view, but not read the files and of course when i change to 644 i could read the files.
the users gid on both linux and freenas is 100.

So i kept fighting myself, but trying to tighten permissions, but not realizing it wasn't recogizing the users group membership.

I was also able to get the home directories working the same way. but it seems (whether browse or --ghost) the drive only mounts when going into it. It is not visible in the nemo file manager, unless entering the home directory of a user from the cli. as a workaround, i created a bookmark which allows quick entry. but i think that is not working properly.

of course, now i can't delete the directory .xdg-volume-info because it says device is always busy ;-)