Best practice for datasets

[JayNil]

I'm planning for a server build (currently acquiring parts) and have read and watched all the stickies/videos/power points on this forum concerning raidz, zfs volumes and datasets.

What I'm wondering is if there's a "best practice" of sort regarding how to set up datasets in relation to user accounts?

My server will have a 4+2 raidz2 setup consisting of 3TB Western Digital Reds.

So:

All disks goes into one big volume called "freenas01".

I have four physical users in my network and one abstract user (of sort). They are:

charles
caroline
laura
albert
htpc

Is it considered best practice to give each physical user in the network their on dataset on the "freenas01" volume?

Is it also wise to make the disk quota "infinite" for these user?

Apart from the physical users, I'm thinking it's useful to give each different media type their own dataset (which to some extent should be read/write/executable by all users):

video
music
photo

I'm also considering giving all the different computers and smartphones on the network their own respective dataset for backup storage.

charles-iphone5
laura-ipad2
albert-galaxynote3
caroline-laptop

and so on.

However, I'm feeling I might be going over the top with all these datasets. How do you guys set it up?
When is a new dataset warranted and when should something just go into a folder in another dataset?

Should the computers/smartphones all go into a dataset called backup and just have their own folders? Is there an added layer of separation/security putting everything in their own datasets?

I feel this is very much like normalizing a database. You can for sure over-normalize and in some circumstances a database becomes unreadable when normalizing too far.

I'd love to hear some comments, ideas and experiences with this.

Thanks,
JayNil

 

[Ericloewe]

Sure, you can make a dataset for each user... But what's the point of doing so? Permissions need to be setup anyway. There is no added security from them being in different datasets.

I do a dataset for each share, which works fine.

 

[JayNil]

Thanks, Eric.

How would those shares be organized, then? Per protocol?

ftp
cifs
nfs
media

I'm thinking maybe each user in a network have a favorite platform and as I understand it, sharing the same data with different protocols is a no-no?
I use Windows/Linux on my workstations. My wife is pure Windows and the kids are a mix of Apple and gaming consoles (Sony, Nintendo).

Should I just create a cifs share and let everyone have their folders under that dataset with permissions set to the subfolders? Would a Linux workstation, Windows laptop, Playstation 3 and an iPad be able to access that same share without causing issues?

I'm trying to grasp how and where permissions should be applied. Not trying to question you. :)

 

[anodos]

Thanks, Eric.

How would those shares be organized, then? Per protocol?

ftp
cifs
nfs
media

I'm thinking maybe each user in a network have a favorite platform and as I understand it, sharing the same data with different protocols is a no-no?
I use Windows/Linux on my workstations. My wife is pure Windows and the kids are a mix of Apple and gaming consoles (Sony, Nintendo).

Should I just create a cifs share and let everyone have their folders under that dataset with permissions set to the subfolders? Would a Linux workstation, Windows laptop, Playstation 3 and an iPad be able to access that same share without causing issues?

I'm trying to grasp how and where permissions should be applied. Not trying to question you. :)

You should think about it in terms of the following:

0) protocol (you should never access the same dataset through multiple protocols
1) simplicity (don't create more than necessary)
2) intelligibility (will you be able to understand which data is located in which dataset / share)
3) snapshot policy (how often do you want to create snapshots and how long do you need to retain them).
4) access controls (who gets access to what)​

Use CIFS. There isn't really a one-size-fits-all approach to (1)-(4). My experience with CIFS is that it is easier to manage access controls at a share level (i.e. having separate "HR", "Accounting", "Projects" shares as opposed to having a "Share" share with subfolders "HR", "Accounting", "Projects", etc)

Also you can control whether a share is advertised on the list of available shares (is browseable). This means you can effectively hide your share full of cat videos.

I would probably point all backups to a single dataset if possible and set a quota on it so that it doesn't fill up your entire zpool.

I try never to give users their own shares. They should be like cattle.

 

[JayNil]

Use CIFS. There isn't really a one-size-fits-all approach to (1)-(4). My experience with CIFS is that it is easier to manage access controls at a share level (i.e. having separate "HR", "Accounting", "Projects" shares as opposed to having a "Share" share with subfolders "HR", "Accounting", "Projects", etc)

Just to clarify here: Do you mean "Share" as a separate dataset here?

Datasets:
"HR", "Accounting, "Projects"...

or

Dataset:
"Data" (with Subfolders "HR", "Accounting", "Projects" being shared individually)

Are "Share" and "Dataset" used interchangeably?

 

[SweetAndLow]

Share and Dataset are not used interchangeably. And it was mentioned that you shouldn't access the same dataset with different protocols, It is more like you shouldn't access the same dataset with different protocols at the same time. You can have multiple protocols access the same data with zero issues but when two people access it at the same time from different protocols you can end up with one person overwriting the data for the other person.

 

[jkh]

Is it considered best practice to give each physical user in the network their on dataset on the "freenas01" volume?

I don't know if it's "best practice" but it's certainly a wise practice, despite what others have said to the contrary. Why? A few reasons:

• You can set the dataset permission for each user such that they own their dataset; you don't have to mess with group access or try to deal with subdirectory-specific access rights for each user, which the UI doesn't cater to well in any case and it's easy to make a mistake with it ("Hey, I'll just recursively set the owner of dataset FOO to owner BAR - that'll fix this issue I'm having. Whoops! I totally forgot that there were multiple users using that dataset! Argh!").
• You can set per-user quotas *later* if user john starts eating more data than they should. Since this is a per-dataset property, you have that option now.
• You can configure a 1:1 relationship between shares and dataset and each user just needs to "log in" to their own share. Again, no fancy group ACLs necessary.

Now, of course, if you want all of these users to collaborate in common areas like "HR" or "Engineering" then those rules change and you're back to creating a dataset per administrative domain, using group membership to allow those users to collaborate.

 

[JayNil]

I don't know if it's "best practice" but it's certainly a wise practice, despite what others have said to the contrary. Why? A few reasons:

• You can set the dataset permission for each user such that they own their dataset; you don't have to mess with group access or try to deal with subdirectory-specific access rights for each user, which the UI doesn't cater to well in any case and it's easy to make a mistake with it ("Hey, I'll just recursively set the owner of dataset FOO to owner BAR - that'll fix this issue I'm having. Whoops! I totally forgot that there were multiple users using that dataset! Argh!").
• You can set per-user quotas *later* if user john starts eating more data than they should. Since this is a per-dataset property, you have that option now.
• You can configure a 1:1 relationship between shares and dataset and each user just needs to "log in" to their own share. Again, no fancy group ACLs necessary.

Now, of course, if you want all of these users to collaborate in common areas like "HR" or "Engineering" then those rules change and you're back to creating a dataset per administrative domain, using group membership to allow those users to collaborate.

A lot of the pros you suggest here are in line with what I was hoping to gain with having 1:1 user/dataset.
How would you go about having one user being able to access all files on all datasets in a setup like that?

Let's say I want user "charles" to have full control over files in the datasets belonging to "caroline", "laura" and "albert"?

 

[Ericloewe]

A lot of the pros you suggest here are in line with what I was hoping to gain with having 1:1 user/dataset.
How would you go about having one user being able to access all files on all datasets in a setup like that?

Let's say I want user "charles" to have full control over files in the datasets belonging to "caroline", "laura" and "albert"?

Easy way: make him the owner.
Hard way: give him permissions manually for all shares

 

[JayNil]

Shouldn't the owner of the dataset be the user for which the dataset was created?

 

[Ericloewe]

Shouldn't the owner of the dataset be the user for which the dataset was created?

Ideally, yes. Now you understand one of the tradeoffs.

Take your pick between the various solutions. Choose what you feel is most logical.

 

[JayNil]

Who likes easy, anyway? :)

For now I think I'll go with the following datasets, users, groups and folders:

• charles (owned by user charles)
• caroline (owned by user caroline)
• laura (owned by user laura)
• albert (owned by user albert)
• media (owned by user htpc), containing folders video, music and photo. (Read/write access for group ingalls)
• backup (owned by user backup), containing folders of each device regularly connected to the network.

Users charles, caroline, laura and albert being members of group ingalls.
Not an overly complicated layout and hopefully intuitive enough to keep track of what data goes into what dataset.

 

[anodos]

Who likes easy, anyway? :)

For now I think I'll go with the following datasets, users, groups and folders:

• charles (owned by user charles)
• caroline (owned by user caroline)
• laura (owned by user laura)
• albert (owned by user albert)
• media (owned by user htpc), containing folders video, music and photo. (Read/write access for group ingalls)
• backup (owned by user backup), containing folders of each device regularly connected to the network.

Users charles, caroline, laura and albert being members of group ingalls.
Not an overly complicated layout and hopefully intuitive enough to keep track of what data goes into what dataset.

A quick caveat: if you use windows client to configure ACLs for your shares, then "ingalls" will need read, write, and modify permissions for your "media" share. Otherwise, it sounds like a good plan.

 

[L]

another quick easy way to do it, is have the dataset be assigned as the home directory when creating the user. It will set the permissions for you

 

[JayNil]

That is a neat solution, Linda!
I'll keep that in mind when setting things up.

Thanks,
JayNil

 

[JayNil]

Hey guys,

Would just like to send a big 'thank you' to everyone for helping me out and answering questions.
Server is assembled and up and running. Currently doing the HDD Burn-in tests as recommended in the How-tos.

Ended up with:

Supermicro X10SL7-F
Intel Xeon E3-1240v3
Fractal Design R4
Samsung 16GB ECC RAM
6 x Western Digital RED 3TB
Kingston DT micro 8GB

No hickups or issues at all so far. I find FreeNAS quite intuitive and my basic knowledge of Linux-based systems gets me through the command line well enough. Read the manual from page to page and I just LOVE the work put into it. Hands down the best manual for a free software I've ever seen.

I'm up at 2:00am getting all worked up over some S.M.A.R.T. tests. I'm loving every second of it. :)

Thanks,
JayNil

 

[willnx]

Let's say I want user "charles" to have full control over files in the datasets belonging to "caroline", "laura" and "albert"?

Hopefully I'm not too late to this conversation to be helpful.

With POSIX - 775 (or whatever for the "everyone" bit)
Owner of the file is the individual owner.
Group Ownership is some group that only "Charles" is a member of - for example, a group named Overlord

Provided "caroline," "laura," and "albert" aren't part of the Overlord group, they will have the "everyone" mode bit permission when it comes to each other's stuff.

With ACLs - much easier! And a reason more granular filesystem permissions were invented!
Share a folder one level up from where everyone's "home" directories will be.
Mount that share via root (or some other user that already has full control)
In Windows Explorer for each of your user's main directory:

Right click -> Properties -> Security -> Edit -> Add -> (enter your user here) -> check name -> click "ok"
*Make sure you're name is highlighted in the top box, then*
In the bottom box, check the "full control" box -> Apply

Woot! ACLs! Not only do you have full control over all the current files, all new files will inherit this permission set too.
With the POSIX method, you'd have to muck about with PAM (which I've only done on Linux boxes, not FreeNAS/ FreeBSD) so that each new file is written with the group ownership = Overlord.

Once you're done, I would recommend getting rid of the Share you created to do this, or at lease lock it down like a mo-fo.


If it helps, here's how my box at home is set up (with the datasets and such):
Vol1
+Berkeley (Unix dataset)
-->So many exports...
+Redmond (CIFS dataset)
-->Two different shares for my only two Windows boxes. Unique permissions for each.
+FTP (Unix dataset)
-->For the crap I always have to re-download when I make a new VM/ build a new PC.

I don't have any quotas set up on these datasets either. I don't care how big any of them get, provided it doesn't fill my NAS 100% and turn it into a big brick.
Instead, I set up a reserve space on Vol1.

To me, the real power/ benefit of multiple datasets are:
More granular snapshots (Clones too)
Different compression preferences
More granular Deduplication control

Using datasets (in my option) for permission control is kind of like using a big wrench to hammer a nail; it works fine, but not it's intended use.


Anyway, I hope you found this to be helpful - or at least not a waste of your time to read.

 

[JayNil]

That is brilliant, willnx! Thanks for your very elaborate answer. I'm sure it'll be useful!

 

[anodos]

With the POSIX method, you'd have to muck about with PAM (which I've only done on Linux boxes, not FreeNAS/ FreeBSD) so that each new file is written with the group ownership = Overlord.

I believe you can just "chmod g+s [folder]". No need to muck around with PAM. Or use "force group". :)

 

[willnx]

I believe you can just "chmod g+s [folder]". No need to muck around with PAM. Or use "force group". :)

Nice! Never thought about using a sticky bit on the group.