Rsync issues migrating files from CentOS6 to TrueNAS12

[jboyce]

Background:
I have a new TrueNAS R20 system that I am setting up as a Samba server for my office. Our existing server is a Dell T610 running CentOS6 and Samba. I have setup the Users/Groups, Pools, and Samba shares on the new system and in the initial tests everything seems fine there. I am now trying to setup and test rsync to migrate directories and files from the old server (sequoia) to the new server (condor).

Problem Summary:
I have managed to get past my initial failures with proper SSH connection and rsync permission denials and have successfully transferred a test set of one of the users home directories. However, I can not access (or even see) the directory in Windows, and I am assuming it is because of a file ownership issue.

Objective:
Migrate all users home directories from the old CentOS server to the new TrueNAS server while keeping the existing owner/group and file permissions as they were on the old system.

Details:
I have rsync setup on TrueNAS (condor) to PULL from CentOS6 (sequoia). If I have Archive, or Preserver Permissions (or both), such as shown below, the directories will transfer but not the files within the directories.

If I unselect the Preserver Permissions option, then rsync will transfer all the directories and files. However, since this rsync task is setup as the root user, the files when they arrive on the TrueNAS system have the owner=root (and a group=allstaff, which I think the group setting will be fine). On the test home directory of the CentOS system that I am using everything has owner=jeffb and group=jeffb. All the permissions for the directories and files on the old system are 0700, and when they show up on the new system after the rsync transfer they are also 0700.

On the TrueNAS system I have a User=jeffb, but I do not have an individual group for the user. All of our users on the new system are included in the group=allstaff. So I am not sure if I have to address adjusting some rsync parameters, or whether I need to make some changes to my user/group setup, or both.

Please let me know if I have missed understanding what my problem is; if there is a bigger issue that I am missing as I am relatively new to TrueNAS. Thanks.

Jeff

 

[Spearfoot]

Background:
I have a new TrueNAS R20 system that I am setting up as a Samba server for my office. Our existing server is a Dell T610 running CentOS6 and Samba. I have setup the Users/Groups, Pools, and Samba shares on the new system and in the initial tests everything seems fine there. I am now trying to setup and test rsync to migrate directories and files from the old server (sequoia) to the new server (condor).

Problem Summary:
I have managed to get past my initial failures with proper SSH connection and rsync permission denials and have successfully transferred a test set of one of the users home directories. However, I can not access (or even see) the directory in Windows, and I am assuming it is because of a file ownership issue.

Objective:
Migrate all users home directories from the old CentOS server to the new TrueNAS server while keeping the existing owner/group and file permissions as they were on the old system.

Details:
I have rsync setup on TrueNAS (condor) to PULL from CentOS6 (sequoia). If I have Archive, or Preserver Permissions (or both), such as shown below, the directories will transfer but not the files within the directories.

View attachment 47767

If I unselect the Preserver Permissions option, then rsync will transfer all the directories and files. However, since this rsync task is setup as the root user, the files when they arrive on the TrueNAS system have the owner=root (and a group=allstaff, which I think the group setting will be fine). On the test home directory of the CentOS system that I am using everything has owner=jeffb and group=jeffb. All the permissions for the directories and files on the old system are 0700, and when they show up on the new system after the rsync transfer they are also 0700.

On the TrueNAS system I have a User=jeffb, but I do not have an individual group for the user. All of our users on the new system are included in the group=allstaff. So I am not sure if I have to address adjusting some rsync parameters, or whether I need to make some changes to my user/group setup, or both.

Please let me know if I have missed understanding what my problem is; if there is a bigger issue that I am missing as I am relatively new to TrueNAS. Thanks.

Jeff

First... make sure the UID of user accounts on your TrueNAS server matches their UID on the CentOS server. So if user jeffb has UID 1200 on CentOS, make sure his UID on the TrueNAS box is 1200, too.

Just for an example, my FreeNAS system's home dataset is owned by root:wheel, with 0770 permissions. Each user subdirectory is owned by the individual user and all belong to the same group ('family', in my case). This works flawlessly for me.

I have found the WinSCP tool handy for changing ownership and permissions 'after the fact' if you find yourself needing to make changes after copying large filesets. Not that you can't do the same thing from the command line; I'm just lazy.

In general, rsync doesn't handle Windows permissions very well, so you may need to tweak things after copying the files.

Good luck!

 

[jboyce]

Thanks for the input. Unfortunately I can't match up the UIDs from the CentOS box as all those users have UIDs in the low 500's, and FreeNAS UIDs started at a base of 1000. Your link regarding the how Windows permissions are (not) handled explained what I was seeing in trying different options. I would get a failure if I used the Archive or Preserve Permissions options, but the Preserver Extended Attributes doesn't fail; although it might not do anything, but at least it doesn't give me a general transfer failure.

I did a little more research after posting (and reading through the rsync man page) and think I have found a solution that is going to work for me (see below).

Adding the --chown=jeffb:allstaff parameter allows me to set the owner and group to what I want when the files arrive at the destination. Reading the rsync man page indicates that the -og parameter is required for the --chown parameter to work. On the source there was a range of permissions with some directories and files set to 0700, 0744, or 0755. At the destination after the transfer all the directory and file permissions have changed to 0700. I am fine with the permission change as it will make everything more consistent across the board.

I also use (and recommend) WinSCP as my left hand tool for observing and understanding what is going on in the process. Now time to sit down and map the transfer process for all my other users and directories. Thanks.

Jeff