SOLVED SMB shares issues on 11.3

[echelon5]

After updating to 11.3, I've partially lost access to samba shares from my Nextcloud instance. I've fixed those by adding "unix extensions = no" to each individual share. It wasn't enough to set this globally.
Proxmox 6.x also lost access to CIFS shares but I'm unable to fix this one.
I'm seeing a new security group in all my shares called "CREATOR GROUP" - what's up with this?

All shares are accessible from my windows machine and FN is joined to a SAMBA4 AD.

Edit:
Regarding my Proxmox issue, I've created a new dataset and share with default settings. I've applied the same ACLs as the previous shares. Initially it wouldn't connect so I've fiddled with aux parameter "unix extensions" and ACL Mode. One of those settings allowed Proxmox to connect to the share. I'll do some more tests.

 

[anodos]

After updating to 11.3, I've partially lost access to samba shares from my Nextcloud instance. I've fixed those by adding "unix extensions = no" to each individual share. It wasn't enough to set this globally.
Proxmox 6.x also lost access to CIFS shares but I'm unable to fix this one.
I'm seeing a new security group in all my shares called "CREATOR GROUP" - what's up with this?

All shares are accessible from my windows machine and FN is joined to a SAMBA4 AD.

Edit:
Regarding my Proxmox issue, I've created a new dataset and share with default settings. I've applied the same ACLs as the previous shares. Initially it wouldn't connect so I've fiddled with aux parameter "unix extensions" and ACL Mode. One of those settings allowed Proxmox to connect to the share. I'll do some more tests.

The CREATOR GROUP entry is a side-effect of this item in the release notes:

The default nfs4:mode was changed from “special” to “simple”. This change is recommended as it synchronizes with Samba defaults and provides a better user experience. If the legacy behavior is required, add the following auxiliary parameter to all SMB shares: nfs4:mode=special. It is important that all shares have the same nfs4:mode setting as they share a common caching backend for SID to ID lookups.

This was a long overdue change, but we held off until a major version change because it could potentially be disruptive and confusing. Proxmox may require SMB1.

 

[echelon5]

The CREATOR GROUP entry is a side-effect of this item in the release notes:


This was a long overdue change, but we held off until a major version change because it could potentially be disruptive and confusing. Proxmox may require SMB1.

Thanks for clearing things up! I remember I once changed that setting too, but I can’t remember how it was last set before the update. Is it possible the group got automatically created even if it was already set to simple? Does it have a negative effect if I remove the group from ACLs?

 

[anodos]

Thanks for clearing things up! I remember I once changed that setting too, but I can’t remember how it was last set before the update. Is it possible the group got automatically created even if it was already set to simple?

No. It's not possible. Changing the parameter has no effect on on-disk permissions.

Does it have a negative effect if I remove the group from ACLs?

That access control entry represents the permissions for the group. If you remove it then the POSIX group will lose permissions. For instance, if your ACL represents 770, you will have potentially the following entries in the Windows SD.
CREATOR-OWNER (inherit-only)
Explicit owner [username] (non-inheriting)
CREATOR-GROUP (inherit-only)

If you remove CREATOR-GROUP, your permissions will become 700 and the group will lose access. The actual group name is not represented in the SD because it can change due to on chown, but chown will not remove the entry. This exactly lines up with `group@` in the NFSv4 ACL.

 

[MikeyG]

I had a similar issue with Nextcloud, but was able to fix it by mounting the SMB shares within the VM Nexcloud is running on and then using "Local" for external storage within Nextcloud instead of an SMB share. I tried adding unix extensions = no to a share, but didn't seem to help. For permissions on the share, I only have an admin group that has full access, not the Creator Owner or Creator Group.

I am however having issues with Veeam backups that I'm wondering if are related. I have a Veeam user, which is part of an admins group that has full rights to the share and folder that Veeam backs up to. Previously in 11.2 that was enough, and logged into Windows as that user (it's an AD user) that user has full rights to the share. But Veeam fails. In order to get it working, I had to give Everyone read access to the share, and then specify the Veeam user as full rights to the backup folder.

This isn't the behavior I'm expecting, as a user part of the group should work correctly. I'm not sure what Veeam is doing differently in accessing an SMB share than say Windows Explorer is, or if it's out of FreeNAS scope, but it seems clear it's some issue with how SMB permissions are done in FreeNAS 11.3.

 

[echelon5]

I had a similar issue with Nextcloud, but was able to fix it by mounting the SMB shares within the VM Nexcloud is running on and then using "Local" for external storage within Nextcloud instead of an SMB share. I tried adding unix extensions = no to a share, but didn't seem to help. For permissions on the share, I only have an admin group that has full access, not the Creator Owner or Creator Group.

I am however having issues with Veeam backups that I'm wondering if are related. I have a Veeam user, which is part of an admins group that has full rights to the share and folder that Veeam backs up to. Previously in 11.2 that was enough, and logged into Windows as that user (it's an AD user) that user has full rights to the share. But Veeam fails. In order to get it working, I had to give Everyone read access to the share, and then specify the Veeam user as full rights to the backup folder.

This isn't the behavior I'm expecting, as a user part of the group should work correctly. I'm not sure what Veeam is doing differently in accessing an SMB share than say Windows Explorer is, or if it's out of FreeNAS scope, but it seems clear it's some issue with how SMB permissions are done in FreeNAS 11.3.

Yeah I used to have SMB shares mounted locally but I preferred NC to mount them directly. There were several reasons but mostly due to indexing which seemed to perform better this way. The issue with NC shares was that libsmbclient tried to mount with SMB 1.0, but that's fixed in more recent versions. The weird thing is, I had 5 folders all mounted with the same dataset and SMB share settings, out of which 2 still worked after the update, only the ACLs were slightly different - then all of them started working with that aux option and no changes to ACLs.

I too had some difficulties with Veeam, but from what I've read in their forums, it seemed to be something on their part. I remember one Veeam O365 version worked only with DNS names and setting samba to strict SMB3 or something like that. I've had issues with Veeam both with FN shares and native Windows shares. I currently have another FN 11.2-U7 as a target for several Veeam Endpoints and Veeam O365 which run fine.

Regarding the Proxmox issue, I don't know how to troubleshoot this - the behavior is inconsistent. I have 3 Prox machines running different 6.x versions. One of them successfully mounted the shares after several tries but the rest still don't work. I did several reboots to Prox and FN. Then I've updated one Prox Host to the same version as the successful one and it still doesn't work.

I'm seeing this in the FN smb logs:

Code:

[2020/02/02 12:36:27.323281,  1] ../../source3/librpc/crypto/gse_krb5.c:417(fill_mem_keytab_from_system_keytab)
  ../../source3/librpc/crypto/gse_krb5.c:416: krb5_kt_start_seq_get failed (No such file or directory)
[2020/02/02 12:36:27.325834,  1] ../../source3/librpc/crypto/gse_krb5.c:417(fill_mem_keytab_from_system_keytab)
  ../../source3/librpc/crypto/gse_krb5.c:416: krb5_kt_start_seq_get failed (No such file or directory)
[2020/02/02 12:36:27.331000,  0] ../../lib/param/loadparm.c:1861(lpcfg_do_service_parameter)
  Global parameter unix extensions found in service section!
[2020/02/02 12:36:27.331781,  0] ../../lib/param/loadparm.c:1861(lpcfg_do_service_parameter)
  Global parameter unix extensions found in service section!
[2020/02/02 12:36:27.332151,  0] ../../lib/param/loadparm.c:1861(lpcfg_do_service_parameter)
  Global parameter unix extensions found in service section!
[2020/02/02 12:36:27.332650,  0] ../../lib/param/loadparm.c:1861(lpcfg_do_service_parameter)
  Global parameter unix extensions found in service section!
[2020/02/02 12:36:27.333382,  0] ../../lib/param/loadparm.c:1861(lpcfg_do_service_parameter)
  Global parameter unix extensions found in service section!
[2020/02/02 12:36:27.377422,  0] ../../source3/smbd/uid.c:448(change_to_user_internal)
  change_to_user_internal: chdir_current_service() failed!

And this on the prox side.

Code:

kernel: [50544.444943] CIFS VFS: validate protocol negotiate failed: -13
kernel: [50544.446498] CIFS VFS: cifs_mount failed w/return code = -2

 

[echelon5]

I've tried everything I could think of and I still can't solve this. In FN 11.2-U7 I had the following:

• Services -> SMB:
  • checked Zeroconf and Obey Pam - rest unchecked; Obey Pam is now dynamic so I've set it to yes/no in aux params, no change;
  • min protocol = SMB2 - tried multiple settings with min/max SMB2,3 or none both on FN and on the hosts smb.conf;
  • nfs4:mode=special/simple;
• Share settings:
  • checked only Browsable to network clients;
  • aux settings initially empty; tried combinations nfs4mode, obey pam and unix extensions;
  • VFS Objects: streams_xattr, zfs_space, zfsacl; unchanged
• DATASET
  • ACL mode passthrough vs restricted;
• Other:
  • created fresh dataset with defaults;
  • reset ACLs on previous datasets;

I believe the issue lies with AD, something related to the kerberos error. When I set the folder ACL to Everyone, I can access shares from Proxmox and Nextcloud, when I remove Everyone, I lose access. Accessing same shares with same credentials from Windows/Mac seems to work ok. There was a brief moment when I couldn't access the shares with an admin account and had to login to Windows again, but that might be a windows problem.

I'm trying some stuff with AD but something doesn't work as it used to.

 

[echelon5]

I've rolled back to 11.2-U7 and shares are working as before. I believe the culprit is this error that doesn't show up in 11.2-U7:

Code:

change_to_user_internal: chdir_current_service() failed!

I still think there's something with AD, because I couldn't leave and rejoin the domain on 11.3. It threw all kinds of errors when leaving and one specific error when rejoining - something about SASL (I forgot to save it).

My SAMBA4 AD is fairly simple and all I'm using it just for FreeNAS ACLs. I currently have a CA certificate which is imported into FN, Encryption Mode: TLS, Kerkeros realm to my domain, backend: rid, SASL Wrapping set to Sign and that's it. In 11.3 I've tried disabling Validate Certificate and switching SASL Wrapping to Seal, and other stuff and I still got that SASL error.

 

[anodos]

In 11.3 we default to performing sasl-gssapi binds (signed with kerberos). Samba DCs by default don't allow these over encrypted transport. You can either choose to set SSL to "OFF" or set "ldap server require strong auth = allow_sasl_over_tls" on the Samba DC.

I could have the LDAP client automatically downgrade to performing a simple bind in this situation, but it would require storing service account credentials (potentially for a user with elevated privileges in the AD environment) which I'd prefer not to do.

 

[echelon5]

In 11.3 we default to performing sasl-gssapi binds (signed with kerberos). Samba DCs by default don't allow these over encrypted transport. You can either choose to set SSL to "OFF" or set "ldap server require strong auth = allow_sasl_over_tls" on the Samba DC.

I could have the LDAP client automatically downgrade to performing a simple bind in this situation, but it would require storing service account credentials (potentially for a user with elevated privileges in the AD environment) which I'd prefer not to do.

Thanks a lot! I've set allow_sasl_over_tls and AD seems to work now. My issue persists though with same error as before.

 

[anodos]

Thanks a lot! I've set allow_sasl_over_tls and AD seems to work now. My issue persists though with same error as before.

Code:

[2020/02/02 12:36:27.377422,  0] ../../source3/smbd/uid.c:448(change_to_user_internal)
  change_to_user_internal: chdir_current_service() failed!

^^^
This almost always means a permissions error in the path to the share. Users need execute across the full path to the share. You can use the ACL manager in 11.3 to add an explicit "TRAVERSE" ACE for domain users.

 

[echelon5]

Code:

[2020/02/02 12:36:27.377422,  0] ../../source3/smbd/uid.c:448(change_to_user_internal)
  change_to_user_internal: chdir_current_service() failed!

^^^
This almost always means a permissions error in the path to the share. Users need execute across the full path to the share. You can use the ACL manager in 11.3 to add an explicit "TRAVERSE" ACE for domain users.

User is in a group with Full Control for the share, including traverse. Wouldn't this be an issue on 11.2 too?

 

[anodos]

User is in a group with Full Control for the share, including traverse. Wouldn't this be an issue on 11.2 too?

Post full testparm output.

 

[echelon5]

Post full testparm output.

I've changed some folder names.

Code:

Load smb config files from /usr/local/etc/smb4.conf
Loaded services file OK.
Server role: ROLE_DOMAIN_MEMBER

Press enter to see a dump of your service definitions

# Global parameters
[global]
    aio max threads = 2
    allow trusted domains = No
    bind interfaces only = Yes
    disable spoolss = Yes
    dns proxy = No
    domain master = No
    enable web service discovery = Yes
    kerberos method = secrets and keytab
    kernel change notify = No
    load printers = No
    local master = No
    logging = file
    max log size = 51200
    nsupdate command = /usr/local/bin/samba-nsupdate -g
    preferred master = No
    realm = SUBDOMAIN.DOMAIN.XYZ
    restrict anonymous = 2
    security = ADS
    server min protocol = SMB2_02
    server role = member server
    server string = FreeNAS Server
    template shell = /bin/sh
    unix extensions = No
    username map = /usr/local/etc/smbusername.map
    username map cache time = 60
    winbind cache time = 7200
    winbind enum groups = Yes
    winbind enum users = Yes
    winbind max domain connections = 10
    workgroup = SUBDOMAIN
    idmap config *: range = 90000001-100000000
    idmap config SUBDOMAIN: range = 20000-90000000
    idmap config SUBDOMAIN: backend = rid
    fruit:nfs_aces = No
    idmap config * : backend = tdb
    allocation roundup size = 0
    directory name cache size = 0
    dos filemode = Yes
    include = /usr/local/etc/smb4_share.conf


[share1]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/shaare1
    read only = No
    vfs objects = shadow_copy_zfs zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share2]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share2
    read only = No
    vfs objects = shadow_copy_zfs zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share3]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share3
    read only = No
    vfs objects = shadow_copy_zfs zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share4]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share4
    read only = No
    vfs objects = shadow_copy_zfs zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share5]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share5
    read only = No
    vfs objects = zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share6]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share6
    read only = No
    vfs objects = shadow_copy_zfs zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share7]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share7
    read only = No
    vfs objects = shadow_copy_zfs zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[Proxmox]
    aio write size = 0
    ea support = No
    level2 oplocks = No
    mangled names = illegal
    oplocks = No
    path = /mnt/data/Proxmox
    read only = No
    strict locking = Yes
    vfs objects = zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share9]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/roland/share9
    read only = No
    vfs objects = zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share10]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share10
    read only = No
    vfs objects = shadow_copy_zfs zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[Temporary Files]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/temporaryfiles
    read only = No
    vfs objects = zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share12]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share12
    read only = No
    vfs objects = zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[testprox]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/testprox
    read only = No
    vfs objects = shadow_copy_zfs ixnas fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[timemachine]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/timemachine
    read only = No
    vfs objects = zfs_space zfsacl fruit streams_xattr
    fruit:time machine max size = 500 G
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:volume_uuid = d51f34be-86cd-4ac5-885c-548fa9a1c6f6
    fruit:time machine = yes
    fruit:resource = stream
    fruit:metadata = stream


[share15]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share15
    read only = No
    vfs objects = zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream


[share16]
    aio write size = 0
    ea support = No
    mangled names = illegal
    path = /mnt/data/share16
    read only = No
    vfs objects = zfs_space zfsacl fruit streams_xattr
    nfs4:acedup = merge
    nfs4:chown = true
    fruit:resource = stream
    fruit:metadata = stream

 

[anodos]

What's the output of "getfacl /mnt/data"?

 

[echelon5]

What's the output of "getfacl /mnt/data"?

Code:

# file: /mnt/data
# owner: root
# group: wheel
            owner@:rwxp--aARWcCos:-------:allow
            group@:r-x---a-R-c--s:-------:allow
         everyone@:r-x---a-R-c--s:-------:allow

file:/mnt/data2
# owner: root
# group: wheel
            owner@:rwxp--aARWcCos:-------:allow
            group@:r-x---a-R-c--s:-------:allow
         everyone@:r-x---a-R-c--s:-------:allow

This has been like this for a long time. I can't remember ever setting ACLs on /mnt/data

 

[anodos]

Set an auxiliary parameter log level = 1 auth_audit:5. Restart SMB service, repeat problem, and PM me a debug.

 

[echelon5]

SOLVED - I think.
First, AD wasn't working - check @anodos post #9

Second, shares worked properly in Linux/Windows, but not in Nextcloud and Proxmox WebUIs.
There are some changes in the default /usr/local/etc/smb4.conf settings. My problem was with username map = /usr/local/etc/smbusername.map. Removing that line made everything work even though the file was empty.

To make the change permanent: Services - SMB - aux parameter: username map = /usr/local/etc/nonexistingfile.map

It'll probably bite me in the behind sometime later, but for now things seem ok.