Sharing (NFS / SMB) is going to kill me...

[TranceKat]

Hi All,
I've spent far too many hours trying to get sharing to work properly with permissions.. I turn to the experts for help (that's you).

Here is my setup:
ESXi 6.7 running FreeNAS 11.3-U3.2 as a VM (Absolutely no issues so far)
Mixed OS environment with both Windows 10 and various flavors of Linux looking to access network shares.
All datasets have been created as SMB shares (I learned this the hard way).

For mounting an NFS share on an apline linux machine, I have the same group name and group ID on both systems.

Code:

dochydra:/mnt/abyss/tmp# id
uid=0(root) gid=0(root) groups=0(root),0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video),102(docker),1001(trancekat),1002(mediamaven)

Here is my mount string from /etc/fstab:

Code:

dochydra:/mnt/abyss/tmp# cat /etc/fstab
UUID=2d558eee-4f24-4706-8619-f2a3293e93c4       /       ext4    rw,relatime 0 1
UUID=e2dd2a43-05a5-4775-9164-9b6ab50da577       /boot   ext4    rw,relatime 0 2
UUID=b2061457-ae5c-416a-8f19-3679c2906525       swap    swap    defaults        0 0
/dev/cdrom      /media/cdrom    iso9660 noauto,ro 0 0
/dev/usbdisk    /media/usb      vfat    noauto  0 0
#//192.168.100.18/Abyss /mnt/abyss cifs credentials=/etc/samba/credentials,gid=1002,iocharset=utf8,_netdev 0 0

192.168.100.18:/mnt/abyss /mnt/abyss nfs nfsvers=4,defaults 0 0
dochydra:/mnt/abyss/tmp#

I mount with sudo mount -a. You will notice an smb mount command commented out. I was able to get SMB to work (painfully), but several containers on the remote alpine linux system break when I try to use smb with them for config storage.

As root on the remote alpine linux system, I can mount the share, and I can see the datasets under that share, but they have strange numbers for owner:group, and I cannot see anything underneath them.

Code:

dochydra:/mnt/abyss# ls -alh
total 201K
drwxrwx--x   10 42949672 42949672      10 Jun 16 14:46 .
drwxr-xr-x    3 root     root        4.0K Jun  9 15:56 ..
drwxrwx---    5 42949672 42949672       5 Jun 16 14:23 applications
drwxrwx---    4 42949672 42949672       6 May 26 22:34 backup
drwxrwx---    6 42949672 42949672      40 Jun 15 12:43 dox
drwxrwx---   27 42949672 42949672      28 Jun 13 14:09 family
drwxrwx---    5 42949672 42949672       5 Jun 15 22:33 media
drwxrwx---    3 42949672 42949672       3 Jun 16 14:48 tmp
drwxrwx---    8 42949672 42949672      17 Jun  3 15:51 work
dochydra:/mnt/abyss# cd tmp/
dochydra:/mnt/abyss/tmp# ls -alh
total 0
dochydra:/mnt/abyss/tmp# mkdir test
mkdir: can't create directory 'test': I/O error
dochydra:/mnt/abyss/tmp#

On my freenas system, this is what it looks like logged in as root:

Code:

Abyss# ls -alh
total 197
drwxrwx--x  10 root  wheel    10B Jun 16 14:46 .
drwxr-xr-x   3 root  wheel   128B Jun 16 12:09 ..
drwxrwx---+  5 root  wheel     5B Jun 16 14:23 applications
drwxrwx---+  4 root  wheel     6B May 26 22:34 backup
drwxrwx---+  6 root  wheel    40B Jun 15 12:43 dox
drwxrwx---+ 27 root  wheel    28B Jun 13 14:09 family
drwxrwx---+  5 root  wheel     5B Jun 15 22:33 media
drwxrwx---+  3 root  wheel     3B Jun 16 14:48 tmp
drwxrwx---+  8 root  wheel    17B Jun  3 15:51 work

The ACL for each of the datasets looks like this:

NFS Settings look like this:

Sharing looks like this:

(Yes, I know setting maproot user to root and group to wheel is bad security, but I am just trying to get it to work at all right now. Will secure once I have some basic functionality)

What am I doing wrong, please? I'm at the end of my rope!

 

[Samuel Tai]

As root on the remote alpine linux system, I can mount the share, and I can see the datasets under that share, but they have strange numbers for owner:group, and I cannot see anything underneath them.

This is the usual behavior for NFSv4. Try NFSv3 to get a more useful uid/gid mapping.

 

[Fredda]

Hmm, I think the observed behavior is still a little bit strange, as usually an incorrect user mapping with NFSv4 results in the
ownership of nobody:nobody for all the groups and files.

To elaborate a little bit:

nfs (v3) does the user mapping by UID/GID. So a file on FreeNAS with ownership root:wheel should show up as root:root on linux.

nfsv4 does the user mapping by Username/Groupname. So a file on FreeNAS with ownership root:wheel should show up as root:wheel on linux.
wheel group usually also exist in Linux, but not with GID 0.

One thing to get nfsv4 working is often overlooked, the domain for server and client has to match, otherwise the usermapping won't work.
The FreeNAS domain is set in Network->Global Configuration.
The Linux NFS domain is set in the /etc/idmapd.conf file.

What you observe looks to me as a very strange mixture of NFS and NFSv4 usermapping gone wrong.
Or maybe your linux has a very strange setting for the Nobody-User/Nobody-Group mapping in the idmapd.conf file.

But as Samuel said, switching to NFS (v3) will probably resolve your problems.

 

[TranceKat]

Thank you both!! I got things working.. though it had more to do with my misunderstanding that I had to add datasets under my top level data set for things to work (I assumed that clicking All Dirs under NFS sharing would allow me to have access to all folders under that top level share).. so for example:

Top level share:
/mnt/abyss
sub datasets shared:
/mnt/abyss/applications
/mnt/abyss/dox
etc.

Now I have full rwx access to those folders from the uid/gid combo I set up on both systems.

The only remaining question I have is how can I prevent the user from seeing the other folders that are not shared. For example:

Code:

dochydra:/mnt/abyss$ ll
ls: ./work: Permission denied
ls: ./family: Permission denied
ls: ./backup: Permission denied
ls: ./test: Permission denied
total 151K  
drwxrwx--x   11 root     users         11 Jun 17 13:39 .
drwxr-xr-x    3 root     root        4.0K Jun 17 16:22 ..
drwxrwx---    6 root     root           6 Jun 17 21:18 applications
drwxrwx---    6 root     root          40 Jun 15 12:43 dox
drwxrwx---    5 root     root           5 Jun 15 22:33 media
drwxrwx---    2 root     root           2 Jun 17 21:22 tmp
dochydra:/mnt/abyss$

Should I not share the top level /mnt/abyss and just continue to share out the sub-datasets and mounting each one individually? (Seems like a kludgy way of doing things).

Thanks again.

 

[Samuel Tai]

AllDirs just means the NFS client can mount both the top-level folder, and any sub-folders as separate mounts. If you don't want some folders available to mount, then you would have to remove the AllDirs export for the top-level folder, and create separate exports for the folders you do intend to clients to mount.

 

[TranceKat]

AllDirs just means the NFS client can mount both the top-level folder, and any sub-folders as separate mounts. If you don't want some folders available to mount, then you would have to remove the AllDirs export for the top-level folder, and create separate exports for the folders you do intend to clients to mount.

Thank you.. I edited my post before I saw your response.

Really appreciate the help.

EDIT: I removed "All Dirs" from the top level share, and am only sharing the 4 folders I want to have available, but I am still seeing the other folders as permission denied when I do ls.

 

[Fredda]

It goes the other way around. E.g., if you only want the applications and media folder be seen on the client and not the others, as the user it does not have permissions to enter anyway.

• Keep the "All Dirs" option enabled
• Instead of mounting the abyss share to /mnt/abyss mount the abyss/applications share to /mnt/abyss/applications and the abbys/media share to /mnt/abyss/media