NFSv4 + Kerberos Problem

[tfili]

Hi,

after upgrade from 11.2-U7 to 11.3 i noticed a problem. The sharing with nfsv4 + krb5 does not work any longer.
I have some experience with this topic, so i found gssd will not started until "Require Kerberos for NFSv4" is active.
This breaks the non kerberos shares, because in the first line /etc/exports the sys entry is missing.

I also test this on a fresh 11.3 installation and found a second problem. I created some datasets for sharing and force kerberos for nfsv4:

pool/share/home
pool/share/home/staff
pool/share/home/staff/user

All three datasets are shared with nfsv4 + krb5 ... I am able to mount pool/share/home on the client and i am able to change into staff but not deeper into user

Is anybody able to confirm this behaviour ?

Regards,

 

[KrisBee]

AFAIK the anwser to your question about those three datasets and what you can access after mounting "pool/share/home" is that each dataset must be considered as a separate filesystem. Check which filesystems are exported on FreeNAS, e.g mount | grep exported . You might be able to enter "staff", but does an "ls" within that directory return an io error on your client?

In the case of a fresh 11.3 install choosing the various options as below appear to create the correct export file as shown by cat /etc/exports

Enable NFSv4	checkbox	Set to switch from NFSv3 to NFSv4. The default is NFSv3.
NFSv3 ownership model for NFSv4	checkbox	Grayed out unless Enable NFSv4 is selected and, in turn, grays out Support>16 groups which is incompatible. Set this option if NFSv4 ACL support is needed without requiring the client and the server to sync users and groups.
Require Kerberos for NFSv4	checkbox	Set to force NFS shares to fail if the Kerberos ticket is unavailable.

Surely, you would expect both gssd and nfsuserd to be running only if you choose NFSv4 + kerberos in the NFS service config. They are not needed if you choose NFSv4 + NFSv3 ownership model.



If you choose NFSv4 + kerberos in the NFS service config and then create a NFS shares on datasets, the export file looks correct, e.g:

Code:

root@freenas[/]# cat /etc/exports
V4: / -sec=krb5:krb5i:krb5p
/mnt/NasPool/mdata -maproot="root":"wheel" -network 192.168.0.0/24
/mnt/NasPool/iplayer -maproot="root":"wheel"
root@freenas

No permission to mount unless client has kerberos ticket etc. If you change the share config of the dataset NasPool/iplayer to security "sys" would you expect it to be mountable without kerberos? . e.g:

Code:

root@freenas[/]# cat /etc/exports
V4: / -sec=krb5:krb5i:krb5p
/mnt/NasPool/mdata -maproot="root":"wheel" -network 192.168.0.0/24
/mnt/NasPool/iplayer -maproot="root":"wheel" -sec=sys
root@freenas

If you remove "require kerberos" from the service config and change the share configs to this example, would you expect the path /mnt/NasPool/iplayer to be mountable without kerberos?

Code:

root@freenas[/]# cat /etc/exports
V4: / -sec=sys
/mnt/NasPool/mdata -maproot="root":"wheel" -sec=krb5:krb5i:krb5p -network 192.168.0.0/24
/mnt/NasPool/iplayer -maproot="root":"wheel" -sec=sys
root@freenas

I don't have kerberos running to test if this last example works for both nfs shares, but the non-kerberos share does mount.

 

[tfili]

Hi and thank you for your reply

each dataset must be considered as a separate filesystem

Yes, of course. Each ZFS dataset must be handled as a separate filesystem, and this is very useful ... not only for nfs sharing also for the backup concept with zfs send / zfs receive or snapshots ...

By the way ... i suppose that this problem is not a special FreeNAS problem, i suppose a problem with the FreeBSD kernel or kerberos libraries.
Years ago i had a similar problem on the client side caused by a kernel update from ubuntu

But i will test this with a fresh FreeBSD 11.3 installation.

And your examples refer only to the case of setting wrong permissions or settings for the nfs sharing.

But our fileservers works pefectly before upgrading to 11.3 ...

I will try to explain my problem step by step on the test-system :

Fresh 11.3 installation

NFS Service config :
+ Enabled NFSv4
- Disabled Require Kerberos for NFSv4

NFS Sharing
Each filesystem only with sys security

Code:

root@freenas[~]# cat /etc/exports
V4: / -sec=sys:krb5:krb5i:krb5p
/mnt/pool/share/home -sec=sys
/mnt/pool/share/home/staff -sec=sys
/mnt/pool/share/home/staff/user -sec=sys

root@freenas[~]# /etc/rc.d/gssd status
Cannot 'status' gssd. Set gssd_enable to YES in /etc/rc.conf or use 'onestatus' instead of 'status'.
root@freenas[~]# /etc/rc.d/gssd onestatus
gssd is not running.

This is OK, because no share uses kerberos, strictly speaking the kerberos secs wouldn't be needed.
On a 11.2-U7 gssd is running anyway.
By the way, in this case, without kerberos, accessing subshares deeper two levels working without problem !

Then i activated the kerberos sec for the shares :

Code:

root@freenas[~]# cat /etc/exports
V4: / -sec=sys:krb5:krb5i:krb5p
/mnt/pool/share/home -sec=krb5
/mnt/pool/share/home/staff -sec=krb5
/mnt/pool/share/home/staff/fili -sec=krb5

root@freenas[~]# /etc/rc.d/gssd onestatus
gssd is not running.

So the share could not be reachable with kerberos security

NFS Service config :
+ Enabled NFSv4
+ Enabled Require Kerberos for NFSv4

Code:

root@freenas[~]# cat /etc/exports
V4: / -sec=krb5:krb5i:krb5p
/mnt/pool/share/home -sec=sys:krb5
/mnt/pool/share/home/staff -sec=sys:krb5
/mnt/pool/share/home/staff/fili -sec=sys:krb5

root@freenas[~]# /etc/rc.d/gssd status
gssd is running as pid 23159.

This (missing sys security) is a comprehensible behaviour (required means required ) but i suppose someone wanted to implement a function in 11.3 that should disable unneeded services and forget the case when a share needs kerberos even though it is not required ....

I don't have kerberos running to test if this last example works for both nfs shares, but the non-kerberos share does mount.

Yes, this is the main problem, kerberos is not common. Only a few users uses a microsoft dc or an univention ucs or similar to secure nfsv4 with kerberos. I do this since 2008 with ZFS and OpenSolaris, OmniOS and later with FreeBSD ... From time to time some update causes big problems and no one are able to confirm :(

 

[KrisBee]

I'm only a home user and in the past created jails for dnsmasq and a kerberos server in order to get NFSv4 + kerberos working with linux clients.
I was looking at this again this morning and the only way I could see to get simultaneous non-kerberos and kerberos NFSv4 mounts working in FN11.3 is to hack the exports file: e.g:

Code:

root@freenas[~]# cat /etc/exports
V4: / -sec=krb5:krb5i:krb5p:sys
/mnt/NasPool/mdata -maproot="root":"wheel" -sec=krb5:krb5i:krb5p -network 192.168.0.0/24
/mnt/NasPool/iplayer -maproot="root":"wheel" -sec=sys

This is with at NFS Service config with + Enabled NFSv4 + Require Kerberos for NFSv4 which ensures gssd is running. Which gives this:

Code:

root@sweep:~# mount -vvv -t nfs   192.168.0.22:/mnt/NasPool/iplayer /home/chris/NFS
mount.nfs: timeout set for Wed Feb 12 09:55:35 2020
mount.nfs: trying text-based options 'vers=4.2,addr=192.168.0.22,clientaddr=192.168.0.201'
mount.nfs: mount(2): Protocol not supported
mount.nfs: trying text-based options 'vers=4.1,addr=192.168.0.22,clientaddr=192.168.0.201'
root@sweep:~# umount /home/chris/NFS
root@sweep:~# mount -vvv -t nfs   192.168.0.22:/mnt/NasPool/mdata /home/chris/NFS
mount.nfs: timeout set for Wed Feb 12 09:55:48 2020
mount.nfs: trying text-based options 'vers=4.2,addr=192.168.0.22,clientaddr=192.168.0.201'
mount.nfs: mount(2): Protocol not supported
mount.nfs: trying text-based options 'vers=4.1,addr=192.168.0.22,clientaddr=192.168.0.201'
mount.nfs: mount(2): Operation not permitted
mount.nfs: trying text-based options 'addr=192.168.0.22'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 192.168.0.22 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: trying 192.168.0.22 prog 100005 vers 3 prot UDP port 731
mount.nfs: mount(2): Invalid argument
mount.nfs: an incorrect mount option was specified
root@sweep:~#

I suppose the exports file hack could be a simple sed command in a FreeNAS init script? Without the sys added to the V4 line, in linux "/mnt/NasPool/iplayer" can only be mounted as NFSv3. Not had a chance to confirm the situation in FN11.2-U7, but I'm happy to accept it is as you say.

I think you should raise this as bug/issue/feature request.

 

[tfili]

Yes, the exports file hack is really simple, adding the :sys at the first line and doing a /etc/rc.d/mountd reload :)

But before i am writing a bug report i will try to reproduce the other problem with the subshares on a plain FreeBSD

 

[tfili]

Short update

The problem with the subshares is a kind of solved ... maybe a permission problem on the filesystem.
By the way, i found some mysterious behaviour when setting permissions / ACLs through the webfrontend
But i will do some additional tests

 

[KrisBee]

@tfili If the odd behaviour is in FN11.3, then I'm not surprised as you have a new ACL editor function and new vfs modules ( for smb at least) with bugs to iron out.