SMB Windows Domain Users Access Denied after 11.3 Update

[geoff.jukes]

@anodos I was able to SSH and create a file as myself.
Right now, everything is working (I've been changing ACLs, tweaking configs etc). I'm not going to touch *anything* and wait.

 

[geoff.jukes]

@anodos With no changes to the datasets ACLs, Samba config, or anything, one client is not getting "Permision Denied" errors.

I have a simple test that I run on 15 Linux machines, and only 1 is getting the issue. To add to the mix, that one is the odd one out, running Centos7 - but it was working just fine earlier today. Rebooting the client and restarting Samba, gets me a 50/50 shot at it working again for a while.

We are investigating an increase in CPU utilization on our DCs that coincides with the FreeNAS update, but may be unrelated (pulling all threads).

These are the Auxiliary parameters we have set for Samba (and have had for a long time):

Code:

unix extensions = no
ea support = no
store dos attributes = no
map archive = no
map hidden = no
map readonly = no
map system = no
read raw = yes
write raw = yes
max xmit = 262144
getwd cache = yes
write cache size = 262144
aio read size = 16384
aio write size = 16384
socket options = TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=2097152 SO_SNDBUF=2097152 IPTOS_THROUGHPUT

I have also enabled SMB1 support for the old Linux servers.

So far, I am unable to clearly identify a set of steps to reproduce or resolve the issue. The most common problem client is the Centos machine connecting to a specific share - but that is the only share it uses, so :shrug:

I'll keep monitoring it....

 

[geoff.jukes]

I wish there was an edit button...

one client is not getting "Permision Denied" errors

should have read

one client is now getting "Permision Denied" errors

 

[anodos]

Remove the auxiliary parameters.

 

[geoff.jukes]

@anodos Without `unix extensions = no` all Linux clients fail.
Adding *only* `unix extensions = no` they all work *except* the Centos7 client.
I'm not comfortable removing all of them (unless you say they are no longer needed). We had extremely poor performance before adding most of them (ea support, dos attributes, etc).
Regardless - removing them does not fix the issue.

 

[anodos]

@anodos Without `unix extensions = no` all Linux clients fail.
Adding *only* `unix extensions = no` they all work *except* the Centos7 client.
I'm not comfortable removing all of them (unless you say they are no longer needed). We had extremely poor performance before adding most of them (ea support, dos attributes, etc).
Regardless - removing them does not fix the issue.

Try mounting with 'noperm' on the problematic CentOS client. Directory listing performance issues should be mitigated in new defaults (vfs_ixnas) without requiring explicitly disabling the DOS modes. Try creating new "SMB" dataset and SMB share using GUI and test performance of new default.s

 

[geoff.jukes]

@anodos No difference. AutoFS already mounts with `noperm`:

Code:

//freenas.domain.local/loc on /smb/freenas.domain.local/loc type cifs (rw,relatime,vers=default,cache=strict,username=loc.worker,domain=DOMAIN,uid=0,forceuid,gid=0,forcegid,addr=172.16.1.25,file_mode=0755,dir_mode=0755,soft,nounix,mapposix,noperm,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)

I'm exploring the impact of the`file_mode` and `dir_mode` values. But as a reminder - this is how it is mounted whe all is working just fine - nothing changes and yet I start getting the permission errors.

 

[geoff.jukes]

(as an aside, I removed all auxiliary parameters *except* `unix extensions = no` . Directory performance is wonderful. Thank you for simplifying that!.... but the issue still persists.)

 

[geoff.jukes]

another aside, I have now modified all my shares to confirm to the new "default".

From this:

To this:

 

[geoff.jukes]

@anodos I set a simple loop running on the Centos server.

In the loop I mount the problematic share, echo the date, attempt to touch a file, then umount it.

Code:

while true; do mount -t cifs //freenas.domain.local/loc /mnt/test -o credentials=/etc/passwd.smb,iocharset=utf8,uid=1000,gid=100,dir_mode=0775,file_mode=0775,noperm; echo $date; touch /mnt/test/dtb/stuff && echo 'OK'; umount /mnt/test; sleep 5; done

After a while, it started working again...

Nothing changed while this loop was running.

 

[geoff.jukes]

(I didn't spot the date wasn't changing. shame, but doesn't change the result).

 

[geoff.jukes]

@anodos Have there been any developments/insights?

 

[geoff.jukes]

Well, in addition to the random "permission denied" I am also seeing issues with file locks not being released. Something is definitely not right @anodos, and I'm happy to help debug.
I've set all shares and pools to the "new deaults" and taken out all ofour legacy "optimizations". The only thing left are network and system tunables. I'll take a look at those next, but I'm not sure what to change or remove.

 

[geoff.jukes]

Just for info, I have also noticed some strangeness/differences with ACL permissions in 11.3 so I am watching this thread very closely. I won't confuse matters with exactly what I've seen as it all sounds very similar but I just wanted to say I think it needs some close attention.

May you wouldn't mind chiming in @Johnny Fartpants

 

[geoff.jukes]

and now I can confirm that random Windows clients are getting Permission Denied errors, connecting to the other FreeNAS server. Which rules out a Linux issue, and a server-specific issue.
All that's left is an upgrade issue.
The most common symptom with the Windows clients, seems to be related to sessions and file locks not being released (i.e. "You need permission from geoff.jukes to perform this action" when I don't have the file open anywhere, but did have it open a while ago.)

 

[Johnny Fartpants]

May you wouldn't mind chiming in @Johnny Fartpants

I need to do more testing before I'm in a position to provide useful feedback. All of my production systems are still on 11.1 or 11.2 so my experience with 11.3 is very little indeed and just testing on a single system. I will try and carry-out more tests over the coming weeks and months and feedback where I feel necessary but as I said at a glance there has been significant changes in ACL behavior and I'm still unsure as of yet if its just a learning curve or if there is actually some underlying issue.

 

[thomisus]

Hi All, i have the same problem regarding random access denied errors and "chdir_current_service() failed!" . I figured that they are not random. this is my situation:

- I have my FREENAS joined to active directory. Netbios name is "FREENAS" , active directory is "ad.*****.network" ( this is a valid domain in split DNS configuration ).
- DNS are correctly pointing and resolving, so "nslookup freenas" and "nslookup freenas.ad.*****.network" are pointing to the same address
- I have created a "test" dataset /mnt/tank/test ( set share type SMB )
- I have smb shared "/mnt/tank/test" with default options (ixnas, streams_xattr)
- I set permissions as follow:
File Information
* user: *****\administrator
* group: *****\domain admins
Access Control List:
* owner@ , allow, basic, full control , basic, inherit
* group@ , allow, basic, full control , basic, inherit
* group: *****\production , allow, basic , full control, basic ( this is an active directory security group )

My user is part of the default "Domain Users" group and "Production" group

The problem is that if i connect to the share using \\freenas\test then i can't connect using \\ip_address\test\ AND \\freenas.ad.*****.network\test
The opposite it's true" if I connect using \\ip_address\test\ AND \\freenas.ad.*****.network\test , I can't connect using \\freenas\test.

[2020/02/11 17:02:53.229863, 3] ../../source3/smbd/service.c:157(chdir_current_service)
chdir (/mnt/tank/test) failed, reason: Permission denied
[2020/02/11 17:02:53.229953, 0] ../../source3/smbd/uid.c:448(change_to_user_internal)
change_to_user_internal: chdir_current_service() failed!

But here it is what i have found: if i change ACL group *****\production to ******\Domain Users , I can correctly login using \\freenas\test , \\freenas.ad.*****.network\test , \\ip_address\test\
No errors.

Looking at active directory groups the only difference is that "Domain Users" group is member of "Users" group ( this is the default built-in local-to-dc group ) and my "Production" group is not member of "Users" group. If i make "Production" meber of "Users" connect error goes away.

 

[thomisus]

On more thing.. If I add to the ACL a local freenas user, he can connect without issues.

 

[anodos]

On more thing.. If I add to the ACL a local freenas user, he can connect without issues.

If this is an issue post upgrade, you might have a caching issue in winbind (stale entries for CREATOR-OWNER and GROUP). Try running "midclt call idmap.clear_idmap_cache"

 

[thomisus]

No, i have reinstalled everything from scratch re-importing the pool