User/Group right management in apps: example of syncthing

[Pickaboo]

Hi everyone, I am new here.
For a new post I will be extensive, sorry for that.

First, I am very impressed with the work done by the devs of iX-Systems, the platform of TrueNAS being ported to Linux is a very appealing to me.
And while I was looking at FreeNAS as a too difficult alternative to shift to this platform, because it was based on BSD, that arrival of SCALE is a benediction for curious people like me.
Congratulations to all of them!

However, after a day "tinkering" with the system, I realized there is a significant hurdle with apps management. Considering how this will be the most attractive feature of the platform in the coming years, I would like to share some thoughts here.
This is truly a bummer to not have a clear understanding of how apps and permissions are managed within the datasets.
It makes the whole experience of server management a hurdle.
I don't want to sound upset in any way. It should be something that is managed in the onset of the system, applications and so on.
I am new to the forum, but I have a experience in linux, so I can help clarify this.

So here are the observations I can relate.
Syncthing is using the user "apps" (UID and GID=568) to operate.
apps is not part of "builtin-users" group, that is a shame, I think they should, more on that later.
Files created and managed by syncthing are hence attributed to the apps user (UID=568) and to the apps group (GID=568).

When files are created by users, they have a GID of 1001, and since apps is lower than user's files (568<1001) user cannot access those files...
That must change. On related example, plex was not showing any files synced by syncthing, simply because they were synced by syncthing...
OK, let's move on.

So syncthing should have two levels of permissions:

1. app managements rights (GUID=568)
2. file management rights (GUID=1001)

And this should be common to all software/apps/containers that deal with user accessible files. Otherwise the management will require tons of chown/chmod to manage them. By being clarified, "tinkering" into file permission will be avoided, and it is the worst thing to make for a appreciated user experience.

To solve the issue this would be giving proper results.
I propose to have two types of datasets: one used for file storage, one used for apps.
The ix-system dataset automatically generated at the first install of a app is properly assigned to apps (GID=568) and the user has no reason to "tinker" there. So all datasets that are not generated by the system should have root:users rights (UID=0:GID=1001)
In files datasets, file management rights will be use, making a clear separation with use interaction files. So all datasets created by users should have (root=0:users=1001) right access. This is not the case.
On the system I set, the main dataset (ZFS "filesystem") has allocated apps as a group (GID=568), and that is not a problem, but also all datasets I created to host my files.
I personally think the main dataset should have GID of 100, because this is root stuff.
That would be a first step.

Then the apps/softs/containers would have to manage two types of rights level, software level (GUID=568) and user files (GUID=1001). That is very common for containerized apps.
There is also the possibility to use the builtin_user group. I think is would be an elegant way of dealing with this, since the apps management does not require the apps user (UID=568) to have any other rights on the user files management.

One very surprising behavior is related to the access management of /st* files by Syncthing. After modifying the access management and GUID parameters of the dataset tree, syncthing was unable to create a /.stfolder in a sub-folder despite being able to sync files in the rest of the tree... It shows that there are two right managements in place, but there are not entirely able to manage operation.

I shall stop here for a first post. And I am pretty sure there are some resources I missed in the documentation.
Please, comment this, say hi or just share your thoughts on this. I believe this is an important part of the user experience that is in question here.

Additional ressources:

Linux Users and Groups

Want to learn more about Linux users and groups? This guide covers the most common user and group management tasks.

www.linode.com

Multi-protocol (NFSv3/SMB) shares - getting the permissions right

I have some datasets that I want to share via NFS to a LAB Kubernetes Pi cluster. I have made the dataset available by NFS and by playing around with perms and NFS mappings I have the dataset 'downloads' available to two deployments and can read and write files - its a bit clunky and I am sure...

www.truenas.com

Cheers

 

[TuxHobo]

I know that I’m adding to this post very late. But I’ve just started to work with SyncThing and came to some realizations that may help the next user.

There are several posts in this community where uses are having problems with file permissions while attempting to use the TrueNAS SyncThing application. It is interesting that most of these requests have gone unanswered.

I found this post because I was having the same trouble: thinking that somehow SyncThing is a multi-user file synchronization tool. It is not! SyncThing is a single user peer to peer file synchronization program.

Others have thought the same thing: it would be great if SyncThing could service multiple users. I found this SyncThing forum feature request. It was not warmly received.

SyncThing is clearly a solid product with a great support community. Once I reoriented my thinking to this peer to peer model, SyncThing fell right into place!

The TrueNAS App setup allows configuration of the user and group UID/GID that SyncThing uses for file access. The default is an unprivileged application ID (587). The same UID/GID is used for SyncThing configuration files and for additional data.

Nothing seemed to work for me when using the default unprivileged user. Either SyncThing could not access the user dataset or SyncThing had to own the files and not the backup user. Or the ugly and insecure 777 file modes.

Assigning the SyncThing UID/GID to my UID/GID (3000 in this case) immediately settled all troubles.
- SyncThing synchronized my desktop properly.
- I have proper access to my files when I ssh to TrueNAS as my user
- SMB shares work as expected for my user.

One thing that could improve the user experience of the TrueNAS SyncThing Application is to make the UID/GID mandatory values. Once I assigned SyncThing to the backup user UID/GID, everything just works.

This solution addresses my immediate home lab problem. But it is not generic. Adding a user requires another SyncThing instance for the new user. This clearly does not scale but it works.

I found this post that describes SyncThing multi user operation. This approach allows multiple users on the same PC to have their own private user space for local SyncThing configuration. It describes the client operation but not the server.

I may be missing something fundamental with the peer to peer approach. Please tell me what I’ve missed. :)

There is an alternative file sync method that does scale. In the past I’ve used the Windows offline file sync tool.
This tool works by making a copy of your remote files on the local machine file system. When files are changed locally, Windows synchronizes the share. If I edit a file from SSH, Windows synchronizes the local files too. If I’m on the go, all files are available locally and edits are synchronized when I connect to my home network again. In the TrueNAS environment it requires an SMB share as the sync target.

For Linux clients I typically mount the SMB share and access it remotely. Windows File sync does the rest.

This works well with Windows clients but I find that offsite backup tools like BackBlaze will not backup remote shares attached to the local machine. This is the primary reason for looking at SyncThing in the first place. If I can find an affordable solution to backup the TrueNAS pools I will probably go back to the windows file sync tool.