Ownership issues migrating data from Windows to FreeNAS...

[SnakeByte]

First, my setup:
FreeNAS-9.3-STABLE-201602031011
Each share is its own dataset.
Each of these datasets have their share type set to "Windows"
I left the Enable atime: Inherit (On) alone.

I've checked both the share and ntfs permissions on each share using compmgmt.msc and made certain that they mirror that of the server I'm migrating away from.
SHARE permissions is Everyone, with full control.
NTFS has Domain Admins, SYSTEM, and Authenticated Users set with Full Permissions.

from a shell on the FreeNas, "net rpc rights list -U DOMAIN\User" shows:

Code:

     SeMachineAccountPrivilege  Add machines to domain
      SeTakeOwnershipPrivilege  Take ownership of files or other objects
             SeBackupPrivilege  Back up files and directories
            SeRestorePrivilege  Restore files and directories
     SeRemoteShutdownPrivilege  Force shutdown from a remote system
      SePrintOperatorPrivilege  Manage printers
           SeAddUsersPrivilege  Add users and groups to the domain
       SeDiskOperatorPrivilege  Manage disk shares
           SeSecurityPrivilege  System security

However, when issuing this command using the same DOMAIN\User account in an "Run as administrator" cmd.exe shell, I get errors:

robocopy \\windows\deploy \\freenas\deploy /MT /COPYALL /Z /B /R:0 /mir /fft
(/MT means multi-threaded)
(/COPYALL copies all file data, including ACL and owner)
(/Z means restart if network interrupt)
(/B means use "backup" mode, which I assume invokes the SeBackupPrivilege and SeRestorePrivilege user rights)
(/R:0 means don't retry on error)
(/mir means make the destination mirror that of the source)
(/fft means work around the DOS time stamp accuracy issue (nix time stamps are more accurate)

like this:

Code:

2016/02/16 10:05:48 ERROR 1307 (0x0000051B) Copying NTFS Security to Destination Directory \\windows\deploy\CoreLib\
This security ID may not be assigned as the owner of this object.

I take this to mean that robocopy is having trouble changing the owner of that CoreLib folder, since:

Checking the owner of the folder on the windows server shows "DOMAIN\Administrators" as the owner.
Checking the owner of the folder on the freenas server shows "DOMAIN\User" as the owner.

If I right click on the folder on the freenas share and attempt to change the owner to "Administrators" I'm given the error: "Unable to set new owner on CoreLib. You do not have the Restore privilege required to set this user/group as owner"

This is getfacl's output for CoreLib:

Code:

getfacl CoreLib
# file: CoreLib
# owner: DOMAIN\User
# group: DOMAIN\domain users
    group:90000005:r-x---a-R-c---:fd----:allow
group:DOMAIN\domain admins:rwxpDdaARWcCo-:fd----:allow
    group:90000006:rwxpDdaARWcCo-:fd----:allow

ls -la shows this:

Code:

d---------+  2 DOMAIN\User     DOMAIN\domain users          5 Dec  8 11:59 CoreLib/

What am I doing wrong?

 

[nightshade00013]

No need to set permissions to "windows" on the datasets for one. It makes it nearly impossible to chmod to fix read write permissions which also could be a problem since no one shows to have read write or execute permissions on the directory in question. So I would suggest just leaving Unix permissions since CIFS will deal with anything that is needed and CIFS doesn't really care about the windows permission structures.

When I do my datasets for the shares there is a main one that will have all of the shared data. I place child datasets under that for my media and each type like movies, music, 3d movies, etc. Once that is all done I log in via SSH and chown -R user:group of the main user account I have setup and the group that all users are allowed to read and write with for the main share directory. Once that is done the group that I setup the windows share with all have the ability to read and write and I move data as needed.

You can also setup all the datasets and then run the wizard and it should do basically the same things but since I have moved the storage and set multiple child datasets what I have done just seems to work out easier especially when it comes to giving Plex read write access as well.

 

[SnakeByte]

nightshade00013,

So the docs are wrong?

http://doc.freenas.org/9.3/freenas_storage.html#change-permissions

The Windows “Permission Type” augments traditional Unix permissions with ACLs. Use the Windows “Permission Type” for CIFS shares or when the FreeNAS® system is a member of an Active Directory domain.

This server is joined to an AD domain. I need to use NTFS permissions. It sounds like what you have done is bypass NTFS permissions, which is completely understandable in a home environment, but not a solution here.

 

[nightshade00013]

Correct I am in a home environment and all I had was problems when using the windows permissions when I wanted to allow read write access for jails actually on the FreeNAS, specifically the write access. Add to that multiple child datasets and the inability to chmod or chown anything with windows permissions attached if I were to add another child.

For AD I am not sure I have not had much need to mess with anything that way. I know it may sound like a silly question but you got everything setup and added to an existing AD domain. Once all the changes were made did you reboot the FreeNAS? I have found that when certain major changes to permissions like changing from UNIX to Windows permissions actually does not take effect properly until a reboot is done. I am not sure exactly why but it's almost as if the ARC gets something for the permissions and just wants to hold on to it and until the cache is flushed out things will not work right.

 

[SnakeByte]

Nightshare00013,

Yeah, I joined the AD domain months ago and have had to reboot the unit a few times since then (FreeNas updates, etc).

I've just done some other tests. Apparently I can change the owner to any other user, and even "Domain Admins" without issue. It's just when I attempt "Administrators" that it is causing this issue.

 

[SnakeByte]

I've just verified -- If I change the owner on the source to something other than "Administrators" robocopy works without issue.

Does anyone know why I am unable to assign "DOMAIN\Administrators" as the owner on anything stored on the FreeNAS server? Some broken user mapping issue? Samba limitation?

 

[anodos]

I've just verified -- If I change the owner on the source to something other than "Administrators" robocopy works without issue.

Does anyone know why I am unable to assign "DOMAIN\Administrators" as the owner on anything stored on the FreeNAS server? Some broken user mapping issue? Samba limitation?

This is a bit out of my area of experience. Are you referring to the BUILTIN\Administrators group? You can view its SID etc via "net groupmap list verbose". Then type "net usersidlist" to view the SIDs for your users. You can assign permissions to BUILTIN\Administrators by changing the permissions location from your domain to the local computer.

 

[SnakeByte]

Anodos,

I believe so, yes. When viewing permissions on our windows server through explorer, it appends the domain so I would see it as DOMAIN\Administrators, not BUILTIN\Administrators. When viewing permissions on the freenas server through Windows Explorer, it appends the machine name, so I would see FREENAS\Administrators.

net groupmap list verbose shows this:

Code:

Administrators
        SID       : S-1-5-32-544
        Unix gid  : 90000001
        Unix group: BUILTIN\administrators
        Group type: Local Group
        Comment   :

On a windows server using explorer, All I need to do to assign "Administrators" as owner is choose it from the list. I.E.:
Right click, Properties, Security Tab, Advanced Button, Owner Tab, Edit... Button

From there I'm given a list:

Code:

Administrators (DOMAIN\Administrators)
myself (myself@mydomain.local)

If I choose Administrators and click the "OK" button, everything works fine.

However, if I do the same thing on a FREENAS share the list I'm given is:

Code:

Administrators (FREENAS\Administrators)
myself (myself@mydomain.local)

If I choose Administrators and click the "OK" button, I'm given this error:

Code:

Unable to set new owner on folder. You do no have the Restore privilege required to set this user/group as owner.

If I do the same thing but also click "Replace owner on subcontainers and objects", I get this error:

Code:

An error occurred while applying security information to:
\\freenas\share\folder\file
This security ID may not be assigned as the owner of this object.

That is the same error as when I run robocopy to copy stuff from my existing windows server to the freenas server. Note the list above shows FREENAS\Administrators, so Explorer already knows to look locally instead of in the domain. Just for fun however, I searched for Administrators from "freenas" instead of the domain, but the same error occurred.

I'm unclear as to why I'm shown DOMAIN\Administrators for the windows server, but not for the freenas server. I double checked dsa.msc, and I don't have an Administrators group created in AD.

If I run subinacl on the windows share folder I've been testing it correctly shows BUILTIN\Administrators as the owner:

Code:

R:\>subinacl /file CoreLib

=================
+File R:\CoreLib
=================
/control=0x400 SE_DACL_AUTO_INHERITED-0x0400
/owner             =builtin\administrators

 

[anodos]

I'm able to add permissions for FREENAS\Administrators in the security tab of Explorer. Changing owner to FREENAS\Administrators does not work. I believe this is because "BUILTIN\Administrators" maps as a unix group and not a unix user. For permissions purposes, winbind (with idmap_rid) / nsswitch / whatever doesn't seem to distinguish between Users / Groups, and so you can chown to an AD Group. This does not work for Unix groups (just like if you used chown at the CLI).

Anyway, that's my best explanation (which is only slightly better than random guessing) for the behavior you're seeing.

 

[SnakeByte]

Now I need to fire up a different server with Samba and test to see if I can assign BUILTIN\Administrators as an owner or not.

According to this: https://technet.microsoft.com/en-us/library/cc783530(v=ws.10).aspx

By default, in Windows Server 2003, the owner is the creator of the object. If the creator of an object is a member of the Administrators group, the Administrators group is the owner. The owner can always change permissions on an object, even when denied all access to the object.

This should mean there are a lot of folders owned by Administrators out there if someone is using the system with administrative privs, right? With all the Samba users out there now who has migrated data from their windows servers, why can't I find anyone else running into this very specific situation when I Google for it?

Do you know how can I verify that "BUILTIN\Administrators" is indeed a unix group? /etc/groups doesn't have administrators listed.

According to /usr/local/etc/smb4.conf, it seems like it isn't a unix group?

Code:

    idmap config *: backend = tdb
    idmap config *: range = 90000001-100000000

net groupmap list shows the builtin\Administrators account as having 90000001

however, there's also this in the config file:

Code:

    idmap config DOMAIN: backend = rid
    idmap config DOMAIN: range = 20000-90000000

Does the backend designate what is or is not a unix group/user?

 

[SnakeByte]

If I crank up the logging to "full" for samba, I get this when attempting to change ownership of the "CoreLib" folder to BUILTIN\Administrators:

Code:

[2016/02/16 15:33:02.077685,  3] ../source3/smbd/vfs.c:1137(check_reduced_name)
  check_reduced_name [CoreLib] [/mnt/trunk/MM/deploy]
[2016/02/16 15:33:02.077890,  3] ../source3/smbd/vfs.c:1267(check_reduced_name)
  check_reduced_name: CoreLib reduced to /mnt/trunk/MM/deploy/CoreLib
[2016/02/16 15:33:02.078111,  3] ../source3/smbd/dosmode.c:163(unix_mode)
  unix_mode(CoreLib) returning 0666
[2016/02/16 15:33:02.080039,  3] ../source3/smbd/posix_acls.c:1204(unpack_nt_owners)
  unpack_nt_owners: unable to validate owner sid for S-1-5-32-544
[2016/02/16 15:33:04.251911,  3] ../source3/smbd/service.c:1130(close_cnum)
  192.168.0.119 (ipv4:192.168.0.119:58406) closed connection to service IPC$

Searching for "unable to validate owner sid for S-1-5-32-544" brings up only a few pages (all over a decade old!) -- some of which hint at what you mention-- sid/gid confusion when stuff is passed to posix_acls.

This thread is interesting:
https://lists.samba.org/archive/samba-technical/2006-October/050007.html

I guess I'll take this up with the Samba folks and see what comes from it.

 

[anodos]

If I crank up the logging to "full" for samba, I get this when attempting to change ownership of the "CoreLib" folder to BUILTIN\Administrators:

Code:

[2016/02/16 15:33:02.077685,  3] ../source3/smbd/vfs.c:1137(check_reduced_name)
  check_reduced_name [CoreLib] [/mnt/trunk/MM/deploy]
[2016/02/16 15:33:02.077890,  3] ../source3/smbd/vfs.c:1267(check_reduced_name)
  check_reduced_name: CoreLib reduced to /mnt/trunk/MM/deploy/CoreLib
[2016/02/16 15:33:02.078111,  3] ../source3/smbd/dosmode.c:163(unix_mode)
  unix_mode(CoreLib) returning 0666
[2016/02/16 15:33:02.080039,  3] ../source3/smbd/posix_acls.c:1204(unpack_nt_owners)
  unpack_nt_owners: unable to validate owner sid for S-1-5-32-544
[2016/02/16 15:33:04.251911,  3] ../source3/smbd/service.c:1130(close_cnum)
  192.168.0.119 (ipv4:192.168.0.119:58406) closed connection to service IPC$

Searching for "unable to validate owner sid for S-1-5-32-544" brings up only a few pages (all over a decade old!) -- some of which hint at what you mention-- sid/gid confusion when stuff is passed to posix_acls.

This thread is interesting:
https://lists.samba.org/archive/samba-technical/2006-October/050007.html

I guess I'll take this up with the Samba folks and see what comes from it.

I think it can't validate the SID because there is no user associated with it. The SID resolves to the group 'BUILTIN\Administrators'. You can see it's a valid group by 'chgrp "BUILTIN\Administrators" foo'

You can post to the main samba mailing list. They are helpful.

 

[anodos]

Mailing list post is recorded here: http://marc.info/?l=samba&m=145566862307356&w=2

 

[anodos]

Now I need to fire up a different server with Samba and test to see if I can assign BUILTIN\Administrators as an owner or not.

According to this: https://technet.microsoft.com/en-us/library/cc783530(v=ws.10).aspx


This should mean there are a lot of folders owned by Administrators out there if someone is using the system with administrative privs, right? With all the Samba users out there now who has migrated data from their windows servers, why can't I find anyone else running into this very specific situation when I Google for it?

Do you know how can I verify that "BUILTIN\Administrators" is indeed a unix group? /etc/groups doesn't have administrators listed.

According to /usr/local/etc/smb4.conf, it seems like it isn't a unix group?

Code:

    idmap config *: backend = tdb
    idmap config *: range = 90000001-100000000

net groupmap list shows the builtin\Administrators account as having 90000001

Code:

idmap config *:backend = tdb

This sets the idmap backend for "BUILTIN" groups. These are stored in a local tdb file on your freenas server. The idmap range gives the range of UIDs / GIDs assigned to the these users / groups. See here:https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/idmapper.html

however, there's also this in the config file:

Code:

    idmap config DOMAIN: backend = rid
    idmap config DOMAIN: range = 20000-90000000

This designates the idmap backend for your domain users and groups.

 

[anodos]

(/B means use "backup" mode, which I assume invokes the SeBackupPrivilege and SeRestorePrivilege user rights)

Did you try the robocopy command without this switch?

Code:

getfacl CoreLib
# file: CoreLib
# owner: DOMAIN\User
# group: DOMAIN\domain users
    group:90000005:r-x---a-R-c---:fd----:allow
group:DOMAIN\domain admins:rwxpDdaARWcCo-:fd----:allow
    group:90000006:rwxpDdaARWcCo-:fd----:allow

This is odd. Those GIDs should resolve to a UNIX group. Post full output of "net groupmap list verbose".

 

[SnakeByte]

Anodos,

Yes, I did try without the /B switch. I only attempted to use that switch thinking that it was required in this special case of migrating all the file metadata. However, even when I attempt to change owner through Windows, I'm stopped there too, so I don't think this is a robocopy usage issue.

Here's the full output of 'net groupmap list verbose':

Code:

Administrators
        SID       : S-1-5-32-544
        Unix gid  : 90000001
        Unix group: BUILTIN\administrators
        Group type: Local Group
        Comment   :
Users
        SID       : S-1-5-32-545
        Unix gid  : 90000002
        Unix group: BUILTIN\users
        Group type: Local Group
        Comment   :

Man, using a list server is so dark ages. :) I got someone at samba.org to bite, so I shouldn't complain.

Btw, is unix account mapping still necessary? If I do an ls -la on the files, I see my domain users and domain groups as owners already -- as if the filesystem natively is already using AD accounts directly.

Here's a sample:

Code:

----------+  1 MMIA\domain admins  MMIA\domain users   41097216 Oct 17  2011 spark_2_6_3.exe
----------+  1 MMIA\domain admins  MMIA\domain users       9685 Dec 17 14:54 uacinstall.vbs
d---------+  3 MMIA\domain admins  MMIA\domain users          4 Sep  4 10:06 uninstaller/
----------+  1 MMIA\domain admins  MMIA\domain users   16652576 Sep 23 08:45 upd-pcl5-x32-5.9.0.18326.exe
----------+  1 MMIA\domain admins  MMIA\domain users   19786968 Dec 15 14:54 upd-pcl5-x64-6.1.0.20062.exe

 

[anodos]

Anodos,

Yes, I did try without the /B switch. I only attempted to use that switch thinking that it was required in this special case of migrating all the file metadata. However, even when I attempt to change owner through Windows, I'm stopped there too, so I don't think this is a robocopy usage issue.

Man, using a list server is so dark ages. :) I got someone at samba.org to bite, so I shouldn't complain.

Actually, the samba mailing list is probably the best place to get samba help. Jeremy, Rowland, and Volker are pretty much the authoritative sources of information when it comes to samba.

Btw, is unix account mapping still necessary?

Of course. You're making a unix server talk windows.

If I do an ls -la on the files, I see my domain users and domain groups as owners already -- as if the filesystem natively is already using AD accounts directly.

Of course, that's a side effect of winbind (a part of samba) + pam + nss. BUILTIN groups are handled a bit differently.

In the end, it appears that Rowland and I reached the same conclusion - you're trying to do something that isn't possible on a Unix system (setting a group to be the user - in terms of ugo permissions).

Another unrelated point of concern is the raw GIDs listed in the getfacl output for the files you copied. They are within the range for the BUILTIN groups, but your "net groupmap list verbose" output doesn't account. It does not appear that FreeNAS is mapping BUILTIN groups other than "BUILTIN\Users" and "BUILTIN\Administrators" (examples of this would be "BUILTIN\Power Users", "BUILTIN\Backup Operators", etc.).

It might be that windows is upset because the other builtin accounts are missing as well. Perhaps you should file a bug report about this. https://bugs.freenas.org/projects/freenas

 

[SnakeByte]

In the end, it appears that Rowland and I reached the same conclusion - you're trying to do something that isn't possible on a Unix system (setting a group to be the user - in terms of ugo permissions).

Well, technically I'm not attempting to assign a group to a user, Samba is. That's the strange thing -- id -u "DOMAIN\Domain Admins" and id -g "DOMAIN\Domain Admins" both return the same value. That's why it works -- Samba uses the correct user/group type for the correct unix field. The question is, why isn't it doing the same for the BUILTIN\ groups? Windows doesn't see unix group mappings at all. In fact, if Samba is a DC, you /can/ assign BUILTIN\Administrators as a owner. Sigh. In any event, yes, this is a Samba thing, not FreeNas. I appreciate your help.

 

[yois]

Just chiming in here.

In AD, the BUILTIN group is technically not an AD group. The BUILTIN\Administrators group only exists as a local permission set for the Domain Controller itself, since a DC has no local accounts. This group exists so that administrative rights can be assigned to users on the LOCAL DC. See this article:

https://social.technet.microsoft.co...nd-domain-admins-in-ad-2003?forum=winserverDS

So it is possible that mapping to this group won't work since it doesn't really exist as a domain group.

 

[SnakeByte]

Hi Yois.

My understanding is that the BUILTIN\Administrators group exists for all windows machines. You are correct that it is not part of AD, however even a DC has this group -- it's just local to each machine, just like "SYSTEM" is.

See for yourself -- you can assign "Administrators" on any windows machine right now -- joined to a domain or not, and even on the DC itself.

The problem I've ran into seems to be a limitation of Samba itself. It is not a unix filesystem limitation either. Take a look at the conversation I've had with the Samba people here (http://marc.info/?l=samba&m=145566862307356&w=2) to see what I'm talking about. Oddly if Samba is set up as a DC, you /can/ assign BUILTIN\Administrators as owner -- it's just when Samba is a member server, that this limitation is exposed. Maybe you can continue the conversation too -- No one is able to (or unwilling to) answer my last question for why this won't work.