SOLVED SFTP with chroot and no shell

[sindreruud]

Hi!

Version:
FreeNAS 11.2 U6

I have tried my best to search the documentation, this forum and google in general. But I couldn't quite find out how to actually configure this on FreeNAS. I have followed the documentation on FreeNAS, aswell as the FreeBSD man-pages for sshd

I have three users and there will probably be more. Currently they are 'john', 'jane' and 'alex'

What I want to accomplish:

All SFTP users belonging to the group sftponly should have their / be this path:
/mnt/vol1/cloud-storage/chroot

I want all users to have their own personal folder at
/mnt/vol1/cloud-storage/chroot/USERNAME

Only the user has read/write access to their own USERNAME folder.

All users can however read but not write to this folder:
/mnt/vol1/cloud-storage/chroot/public

I have made cloud-storage as a dataset, and given user root rwx, and group wheel r-x access. As i understand it, when you want chroot on a folder, it has to be owned by user root, and it cannot be group writable. Here is the output:
drwxr-x--- 3 root wheel 3 Feb 10 15:59 cloud-storage

Then I made a sub-dataset at /mnt/vol1/cloud-storage named chroot and it has the same permissions:
drwxr-x--- 5 root wheel 5 Feb 10 16:01 chroot

In the chroot dataset there is several sub-datasets owned by the particular user
/mnt/vol1/cloud-storage/chroot/john /mnt/vol1/cloud-storage/chroot/jane /mnt/vol1/cloud-storage/chroot/alex /mnt/vol1/cloud-storage/chroot/public (viewable by all)

The permissions looks like this:

Code:

ls -l /mnt/vol1/cloud-storage/chroot
drw-------  2 alex  sftponly  2 Feb 10 16:00 alex
drw-------  2 john  sftponly  2 Feb 10 16:00 john
drw-------  2 jane  sftponly  2 Feb 10 16:00 jane
drwxr-----  2 root  sftponly  2 Feb 10 16:01 public

Users and the sftponly group is created through FreeNAS UI. All the users belong to the group sftponly that I created. They are added as main group, but I have tried it as Aux group aswell.

Homedirectory is set as /nonexistent
Shell is set as /usr/local/bin/scponly
Initially I tried setting the shell as /usr/sbin/nologin, but this was just creating errors upon connection.

Checking that all users belong to the correct group:

Code:

freenas# id alex
uid=1000(alex) gid=1002(sftponly) groups=1002(sftponly)
freenas# id jane
uid=1002(jane) gid=1002(sftponly) groups=1002(sftponly)
freenas# id john
uid=1003(john) gid=1002(sftponly) groups=1002(sftponly)

And finally on to the sshd_config. I first tried to edit the config via terminal at /etc/ssh/sshd_config

Code:

freenas# nano /etc/ssh/sshd_config
[...]
#Subsystem       sftp    /usr/libexec/sftp-server
Subsystem       sftp    internal-sftp

Match Group sftponly
        ChrootDirectory /mnt/vol1/cloud-storage/chroot
        ForceCommand internal-sftp
        AllowTcpForwarding no
        X11Forwarding no

First of all this config change did not survive a reboot. Second, all the users is logged in to the / of the FreeNAS filesystem, and not into the specified ChrootDirectory path.

Therefor I looked at editing it in the UI instead. I went to Services -> SSH, selected configure, and under the "Extra options" i specified the following:

Code:

Subsystem sftp internal-sftp
Match Group sftponly
ChrootDirectory /mnt/vol1/cloud-storage/chroot
ForceCommand internal-sftp
AllowTcpForwarding no
X11Forwarding no

I can save this, but when I deactive SSH and activate it again (in the UI) a dialog box pops up with this message
SSH service failed to start.

According to the FreeNAS docs regarding 'Extra Options'

Add any additional sshd_config(5) options not covered in this screen, one per line. These options are case-sensitive and misspellings can prevent the SSH service from starting.

And according to the FreeBSD man-page sshd_config(5) regarding 'Match'

Introduces a conditional block. If all of the criteria on the Match line are satisfied, the keywords on the following lines override those set in the global section of the config file, until either another Match line or the end of the file.

So as far as I can see, I have configured it correct. So I am at a complete loss as to why this is not working. I have tried rebooting the entire system aswell, to no avail.

I would like to add that as soon as I have this up and running I will deactivate password login, and use ed25519 keys for auth. In addition i have port 22 blocked on my router. Instead, clients will connect with a high numbered port that I have forwarded to 22 internally.

Any help here is highly appreciated!

 

[sindreruud]

I just realized that I have not checked the permissions on /mnt and /mnt/vol1

I think I have made a huge mistake in my early days of this system when I was not very smart:

Code:

freenas# ls -l /
[...]
drwxr-xr-x   3 root  wheel   128 Feb 10 21:01 mnt
[...]

freenas# ls -l /mnt
total 15
-rw-r--r--  1 root  wheel    5 Oct  1 17:44 md_size
drwxrwxr-x+ 9 root  sindre  10 Feb 10 15:01 vol1

So /mnt/vol1 is owned by a random group I created a long time ago, with rwx permissions. In addition, in the UI, the ACL for the dataset is "Windows".

So questions
1) Do I need to change the ACL from Windows to Unix, and will this create headaches? There was a couple of warnings in the UI for this
2) Can I change the group to wheel, and r-x permissions without it creating headaches? I would not do it recursively, and I only rely on the permissions inside the vol1 dataset, not the dataset itself. I think atleast.

 

[Fredda]

I think this is the mistake you made:

Code:
ls -l /mnt/vol1/cloud-storage/chroot
drw------- 2 alex sftponly 2 Feb 10 16:00 alex
drw------- 2 john sftponly 2 Feb 10 16:00 john
drw------- 2 jane sftponly 2 Feb 10 16:00 jane
drwxr----- 2 root sftponly 2 Feb 10 16:01 public

To enter a directory you need the "x" permssion to do so, which is missing here.
So the user directory should have permission 700 and the public should have 750.
The rest looks OK (IMHO)

 

[sindreruud]

I think this is the mistake you made:

To enter a directory you need the "x" permssion to do so, which is missing here.
So the user directory should have permission 700 and the public should have 750.
The rest looks OK (IMHO)

You are absolutely right, thanks for pointing that out. I have was so adamant about not allowing my users to execute anything on my server that I forgot about permissions on directories. From Wiki:

• The execute permission grants the ability to execute a file. This permission must be set for executable programs, in order to allow the operating system to run them. When set for a directory, the execute permission is interpreted as the search permission: it grants the ability to access file contents and meta-information if its name is known, but not list files inside the directory, unless read is set also.

The effect of setting the permissions on a directory, rather than a file, is "one of the most frequently misunderstood file permission issues".

I have fixed this issue now. However, I still think that the sshd_config should chroot the users too /mnt/vol1/cloud-storage/chroot, even though the directories within the chroot directory had wrong permissions.

I have tested this now, and the SSH service still wont start when I add the "Extra options" from the UI.

Does every folder above /chroot also need to be owned by user root, and not be group writable? In that case, that is where my (hopefully) last mistake is.

I just need to be sure I do not create too many problems by changing the permissions on those directories.

 

[Patrick M. Hausen]

Does every folder above /chroot also need to be owned by user root, and not be group writable? In that case, that is where my (hopefully) last mistake is.

Yes. The path from the root directory all the way down to the chroot directory needs to be owned by root and not writeable by anyone else.

HTH,
Patrick

 

[sindreruud]

Yes. The path from the root directory all the way down to the chroot directory needs to be owned by root and not writeable by anyone else.

HTH,
Patrick

Thank you for confirming. I have now fixed my mistakes, and as far as I can see, the permissions are now correct all the way from / and down to /chroot.

Code:

freenas# ls -l /
drwxr-xr-x   3 root  wheel   128 Feb 10 21:01 mnt


freenas# ls -l /mnt
drwxr-xr-x+ 9 root  wheel  9 Feb 11 10:36 vol1

freenas# ls -l /mnt/vol1
drwxr-x---   3 root         wheel    3 Feb 10 15:59 cloud-storage

freenas# ls -l /mnt/vol1/cloud-storage
drwxr-x---  5 root  wheel  5 Feb 10 16:01 chroot

freenas# ls -l /mnt/vol1/cloud-storage/chroot
total 2
drwx------  2 alex  sftponly  2 Feb 10 16:00 alex
drwx------  2 jane  sftponly  2 Feb 10 16:00 jane
drwx------  2 john   sftponly  2 Feb 10 16:00 john
drwxr-x---  2 root         sftponly  2 Feb 10 16:01 public

However, while applying "Extra options" to the SSH service in the UI, it still gives a error when trying to start the SSH service. I have tested this further by applying less arguments. Right now i tried adding just the following:
Subsystem sftp internal-sftp
And that fails in the same manner.

Removing all the text from "Extra options", and the SSH service starts fine from the UI. What am I missing?

 

[Patrick M. Hausen]

Is there an error message in /var/log/middlewared.log or have you tried service sshd restart on the command line and watch what it complains about?

 

[sindreruud]

Doesn't seem to be anything interesting in the log

Code:

[2020/02/11 11:17:59] (DEBUG) ServiceService._simplecmd():281 - Calling: reload(ssh) --> This was probably after i saved the config
[2020/02/11 11:18:11] (DEBUG) ServiceService._simplecmd():281 - Calling: stop(ssh)   --> Me stopping the service in the UI
[2020/02/11 11:18:36] (DEBUG) ServiceService._simplecmd():281 - Calling: start(ssh)  --> Me starting the service in the UI but failing
[2020/02/11 11:19:49] (DEBUG) ServiceService._simplecmd():281 - Calling: start(ssh)  --> Me starting the service after removing "Extra options" and it succeeds

Now I tried to add the Extra options, without stopping and starting the service manually. The log says
[2020/02/11 11:26:41] (DEBUG) ServiceService._simplecmd():281 - Calling: reload(ssh)
But nothing more.

When trying to connect one of the users to sftp, they are still going to the / of FreeNAS, seeing the entire system.

I then tried service sshd restart the output is:

Code:

freenas# service sshd restart
Cannot 'restart' sshd. Set sshd_enable to YES in /etc/rc.conf or use 'onerestart' instead of 'restart'.
freenas# service sshd onerestart
Performing sanity check on sshd configuration.
No host key files found

Nothing shows up in the /var/log/middlewared.log after this command, but I don't know if it is supposed to either.

I had planned to set up the host key file after I confirm that I have this working, just to make the testing slightly easier. Is this what is stopping me?

 

[Fredda]

It should be service openssh restart. (At least with 11.2)

Also, the sshd config files are located in /usr/local/etc/ssh and not /etc/ssh, they both exist I don't why, but the one in /usr/local/etc is used.

 

[sindreruud]

It should be service openssh status. (At least at 11.2)

Also, the sshd config files are located in /usr/local/etc/ssh and not /etc/ssh, they both exist I don't why, but the one in /usr/local/etc is used.

Thank you! I have been wondering about that, but couldn't find documentation for it. Maybe I didn't look closely enough. Anyway, this proved to be very useful.

Code:

freenas# service openssh status
openssh is running as pid 11706.
freenas# service openssh restart
Performing sanity check on openssh configuration.
/usr/local/etc/ssh/sshd_config line 16: Subsystem 'sftp' already defined.

So apparantly the "Extra options" in the UI appends the arguments to the end of the sshd_config. And the sshd_config already has Subsystem defined. My config is now looking like this:

Code:

Subsystem       sftp    /usr/libexec/sftp-server -l ERROR -f AUTH
Protocol 2
UseDNS no
ChallengeResponseAuthentication no
ClientAliveCountMax 3
ClientAliveInterval 15
NoneEnabled yes
VersionAddendum none
Ciphers +aes128-cbc
Port 22
PermitRootLogin yes
AllowTcpForwarding no
Compression no
PasswordAuthentication yes
PubkeyAuthentication yes
Subsystem sftp internal-sftp
Match Group sftponly
ChrootDirectory /mnt/vol1/cloud-storage/chroot
ForceCommand internal-sftp

So i commented out the first line, and restarted openssh again. Looking good:

Code:

freenas# service openssh restart
Performing sanity check on openssh configuration.
Stopping openssh.
Waiting for PIDS: 11706.
Performing sanity check on openssh configuration.
Starting openssh.

Logging in as sftp user:

Code:

alex@192.168.1.112's password:
Connected to 192.168.1.112.
sftp> ls /
remote readdir("/"): Permission denied
sftp> cd /etc
Couldn't canonicalize: Permission denied
sftp> ls /alex
Can't ls: "/alex" not found
sftp> ls /public
Can't ls: "/public" not found

So now we are really getting somewhere, I seem to be chrooted somewhere, though I am a bit unsure where I actually am now. I cannot ls, and I cannot ls /alex or /public.

Thoughts?

 

[Fredda]

I think this is still the problem:

freenas# ls -l /mnt/vol1
drwxr-x--- 3 root wheel 3 Feb 10 15:59 cloud-storage
freenas# ls -l /mnt/vol1/cloud-storage
drwxr-x--- 5 root wheel 5 Feb 10 16:01 chroot

Your users are not allowed into those directories. You would need at least w+x permissions and the directory
/mnt/vol1/cloud-storage/chroot needs w+rx permissions.

 

[sindreruud]

Currently /mnt/vol1/cloud-storage/chroot has the following permissions
drwxr-x--- 5 root wheel 5 Feb 10 16:01 chroot

So root has rwx and group has r-x. This is to be inline with the no group-writable on the chroot directory or any directory above it.

Inside chroot it looks like this for user alex:
drwx------ 2 alex sftponly 2 Feb 10 16:00 alex

So I thought this would make user alex able to cd /alex and cd /public, but it just outputs Couldn't canonicalize: Permission denied

So I think I am not understanding your post entirely, which permissions do you think is wrong?

Thank you all for the help so far, I am very grateful. Feels like I'm really close to understanding it now.

 

[Patrick M. Hausen]

Make all the directories all the way down to chroot root:wheel 755. They must be readable by the user alex.

BTW: you can chroot each user into his/her own directory by using ChrootDirectory %h and setting the home directory of the user to the desired target. But again all directories including the home *must* be owned by root and 755.
You can then add the first user writeable directory one below that home.

HTH,
Patrick

 

[sindreruud]

I feel stupid now. The permissions for cloud-storage and chroot was 750 and not 755. Changing that obviously worked. Users are now chroot'ed into /chroot as expected, and they only have rwx to their own username directory.

However, doing ls exposes all the username folders. Not really a problem as you can't read or write the content of others directories, but I'm guessing best practice would be not seeing other users directories at all.

Patrick already suggested ChrootDirectory %h, which would be a viable solution. I guess I could also do ChrootDirectory /mnt/vol1/cloud-storage/%u with root:wheel 755 all the way down to %u or %h.

However, how would I go about giving all my users read access to the same public folder? My use case is that I periodically have files/folders that I need my users to download, and putting them in this so-called public directory would be very easy. Is this configurable at all, with chroot?

 

[Patrick M. Hausen]

• Create a separate dataset/directory for the shared dir
• Nullfs mount that into each user's chroot home, e.g. by a post-startup task
• For extra credit: get rid of messing with group permissions, umask and all that stuff by simply giving each user the same numerical user ID - they will then all be able to write in the shared thingy

The last one is perfectly legal in Unix systems, but many people don't know. Since chroot takes care of the user separation, identical UIDs are not a problem.

 

[sindreruud]

Thank you so much Patrick and Fredda! I have now marked this as solved.

I'll look into the nullfs mount solution you provided, but first I need to read up on what that entails. Thanks!

For anyone coming here in the future, I advise you read the entire thread, but here is the summary atleast:

1.
All directories leading from / and all the way down to the chroot you specify in sshd_config needs to have root:wheel owner and 755 permissions. In other words, user root needs rwx, group wheel needs r-x and other/world needs r-x. So ls -l should output drwxr-xr-x. Make sure you check all directories, all the way from / and down to the ChrootDirectory specified in the config.

2.
Editing "Extra options" for SSH in the UI can lead to errors (did for me atleast)
The correct sshd_config to edit is at /usr/local/etc/ssh/sshd_config

3.
To do status, restart, stop or start, the correct command is service openssh status

 

[Patrick M. Hausen]

Hint: mount -t nullfs /real/path/to/public /chroot/user/home/public. Repeat for every user. The target directory must exist before mounting.

 

[Fredda]

For extra credit: get rid of messing with group permissions, umask and all that stuff by simply giving each user the same numerical user ID - they will then all be able to write in the shared thingy

How do you do that? Out of curiosity I tried that in a jail and it did not work. When I tried to create a testuser with an existing UID with the adduser command the created user had the next free UID and not the already existing one.

 

[Patrick M. Hausen]

adduser ... newfangled nonsense! vipw ;) Possibly pw useradd will work, too.

 

[Fredda]

Ugh, vipw, and I thought I was old ....