SOLVED Rsync to linux – ownership options not working.

[ragametal]

As the title says, i’m trying to set an Rsync task to backup my truenas server to an rsync module i have created on a separate linux box (ubuntu 18.xx).

I have successfully established the Rsync connection between the two systems and i created the same user accounts and groups on both systems (with different UID’s and GID’s of course).

The problem that I’m having is that all the files on the target folder have an ownership of root:root with permissions set to 777. What I want is to set the owner of the destination files to be the same as the source files and hopefully assign permissions 770.

Now, I understand that there is a compatibility problem between the dataset ACLs in truenas and the implementation of rsync in linux but i have avoided the options -a; -p & -A as suggested by @Spearfoot in the Post Thread " Impaired rsync permissions support for Windows datasets ".

On truneas I'm passing the auxiliary options for --owner & --group and, while rsync is not giving me any errors, the end result is that all target files are still owned by root:root with 777.

My rsyncd.conf settings (in the linux box) are as follows:

Code:

[socrates]
        path = /media/backup-usb/rsync
        comment = for the rsync backup from truenas server
        read only = false
        timeout = 300
        auth users = socrsync
        secrets file = /etc/rsyncd.secrets

and my settings in truenas are:

The auth user in the rsyncd.conf (“socrsync”) has been added to the sudoers list by adding the line
socrsync ALL=NOPASSED:/usr/bin/rsync

I tried to add the option fake super = yes to my rsyncd.conf but rsync fails if i do that. It also fails if i pass the following options from truenas: --fake-super or -t

I also tried passing the option --super but the end result is the same (no errors but ownership is root:root).

So, do anybody knows how can I rsync to a linux module and retain the original ownership of my files?

 

[winnielinnie]

I can understand forgoing -a (archive mode), -A (ACLs), and -X (xattr), but why leave out -p? Isn't that the entire point in the first place, to preserve the permissions?

(In your example, you're using -X anyways, which might not be relevant to this issue.)

Also, why not preserve timestamps as well?

 

[ragametal]

... but why leave out -p? Isn't that the entire point in the first place, to preserve the permissions?

...why not preserve timestamps as well?

Because rsync fails on me if i set any of those two options. @Spearfoot explained the restriction on the permissions but i i have no clue why "timestamps" fails.

Anyway, there are a bunch of wrong things with this particular rsync setup and i wanted to focus on one problem at a time. This time file ownership.

 

[winnielinnie]

You're also missing uid and gid in your Rsync module on your Ubuntu 18.xx server.

 

[ragametal]

You're also missing uid and gid in your Rsync module on your Ubuntu 18.xx server.

aren't those options to force the ownership of files to a particular user/group on the target folder?

I thought that by using --owner & --group on truenas i was passing the ownership parameters so that the target files matches the ownership of the source files.

Isn't this the case?

 

[winnielinnie]

aren't those options to force the ownership of files to a particular user/group on the target folder?

Isn't that what you want?

I thought that by using --owner & --group on truenas i was passing the ownership parameters so that the target files matches the ownership of the source files.

I thought you said that your TrueNAS and Ubuntu boxes don't share the same uids/gids?

Why not add to your module,

Code:

uid = ragametal
gid = ragametal

Or even just the number, like so,

Code:

uid = 1002
gid = 1002

Then remove --rsync-path, --owner, and --group from your Rsync task on TrueNAS. I'm assuming ragametal on Ubuntu has full write access to /media/backup-usb/

I'm using off-the-cuff names and numbers for the sake of example.

 

[ragametal]

I apologize, i can see how my comment was confusing.
I do have the same uid and gid on both systems. however their id numbers are different.

 

[winnielinnie]

I do have the same uid and gid on both systems. however their id numbers are different.

Linux/Unix doesn't care about the name as much as the numeric value. If ragametal on TrueNAS is 1000, but ragametal on Ubuntu is 1002; even though they share the same "username", they are different users for all intents and purposes of file ownership: 1000 vs 1002

See if you get any closer with the above suggestions on modifying your module (on Ubuntu) to include uid and gid to match the local Ubuntu user's actual number, such as 1002, 1003, or whatever it is.

Try it while also removing the other mentioned options on your TrueNAS Rsync task.

(I'm not sure what type of effect this might have on folders/files that were already written from a previous sync.)

 

[ragametal]

Got it. Thanks for the clarification.
I tried adding the uid and gid to the rsyncd.conf and got the same result.

Your clarification made me research a bit more and, according to the rsync man page, there are two options to match the users/groups of the source system to the users/groups on the target system. These options are --usermap & --groupmap.

I tried passing these options from Truenas (both by their uid names and numbers) but the end result was the same, root is the owner of all the target files.

I'm starting to believe it has to be something on the linux side of things because I'm running out of ideas of things to try on the Truenas side of things.

 

[winnielinnie]

These options are --usermap & --groupmap.

I tried passing these options from Truenas (both by their uid names and numbers) but the end result was the same, root is the owner of all the target files.

How did you write them out in Rsync task under TrueNAS?

Something like...?

Code:

--usermap=truenasrsyncuser:ubuntuuser
--groupmap=truenasrsyncuser:ubuntugroup

Or even...

Code:

--usermap=1002:1000
--groupmap=1002:1000

---

Who did you set to run the Rsync task from TrueNAS? If their uid is say, 1002, and your Ubuntu's user is uid is 1000, you can map 1002:1000.

 

[ragametal]

I actually tried both.

On Truenas, under the Auxiliary parameters i added

Code:

--usermap:1000:1004
--groupmap:1000:1013
--owner
--group
--fake-super
--numeric-ids

I added the last four options as suggested by the rsync man. Same as before, no errors and the owner of the target files was still root:root.

Then, because both systems have user accounts and groups with the same names, i tried

Code:

--usermap:rsyncuser:rsyncuser
--groupmap:group1:group1
--owner
--group
--fake-super

Same outcome. This is how it looks now.

Based on what i have read, these options should have taken care of the ownership problem but they don't in my case. I honestly don't know what am I doing wrong.

 

[ragametal]

I think i have found the problem and it is on the linux box.
I ssh into the linux box with the rsync user and created a file with
touch new file

would you like to guess who was the owner? root:root of course with permissions set to 777.

Therefore, i need to find out why ubuntu is assigning root:root as the owner for all new files in that folder.
I know this is not a linux forum but, does anybody knows what setting would be responsible for this behavior in ubuntu?

 

[winnielinnie]

would you like to guess who was the owner? root:root of course with permissions set to 777.

Wait...

Is this on an external drive, per chance? What filesystem is it? NTFS? FAT32? exFAT? Ext4? XFS? Btrfs?

 

[ragametal]

Yes... it is an external drive that i use just for backup purposes.
Yes... It does have an exFAT file system.

It is permanently attached to the Linux machine.

 

[ragametal]

@winnielinnie
Your response made me try something different. On the linux box, i created a new rsync module with a path pointing to an internal drive with ext4 file system.

I added uid = root in the global settings of the rsyncd.conf file and, on Truenas, I changed my settings as follows:

These settings produce the desired result. All files have the same owner, group, times, and even similar permissions (770) as the source files.

I guess that my real problem is the filesystem on my external hadrdrive (exFAT). Besides formatting this disk to ext4, Would you have a suggestion on how to solve this permissions issue?

The idea behind the exFAT filesystem is to be able to connect this portable drive to a windows machine if i ever needed to access its information and a linux box was not within reach. However, i would gladly give away this convenience for the sake of retaining proper ownership and permissions of my files.

 

[winnielinnie]

Yes... It does have an exFAT file system.

That's why.

exFAT, like FAT32, doe not support native POSIX permissions and ownership.

The cleanest method would be to mount the exFAT filesystem with options that satisfy you while it is connected to the Ubuntu box. It's meaningliness once you umount/unplug the drive, as no such permissions are permanently stored. If you were to plug the exFAT drive into another Linux PC, it would present it as 777 permissions.

Mounting the exFAT filesystem with the following options might help:
uid=luis,gid=socrsync,umask=0007

Assuming you want all files and folders to have 770 permissions, with the Ubuntu user/group as luis/socrsync.

This approach would be much cleaner. You can then use fewer options for the Rsync task and module. Won't need groupmap, usermap, fake-super or any other extraneous options.

However, i would gladly give away this convenience for the sake of retaining proper ownership and permissions of my files.

Then you need to use a filesystem native to Linux, such as Ext4 or XFS.

 

[ragametal]

Honestly, i think it would be more straight forward and simpler if I just change the file system of the external HDD to Ext4 to match the other disks in that system. At least it will be consistent in the sense that all the disks would use the same filesystem type.

@winnielinnie thank you so much for the help and for your constant suggestions of new things to try. I would not be able to figure this out without your help, that's for sure.