Making ACLs and nested datasets work as expected, in the context of snapshots and Samba shares?

[Stilez]

This is a complex area where 4 FreeNAS subsystems overlap - ACLs, ZFS pool structure, ZFS snapshots, and Samba shares. I know what I'd hope to be the expected behaviour, but I'm pretty sure it doesn't work that way. I'm posting this to get a good understanding how it does work, and to learn what I can do to mitigate "unexpected" behaviours (as seen from the perspective of a Windows + Samba end-user).

For discussion, here is a simplified pool setup, as it would be exposed by Samba (files omitted, dirs shown only, recycler + snaps accessible):

Note: this layout is purely an example to aid discussion, it is *not* a representation of my pool, so it isn't useful to ask "why would you want the pool laid out that way?" The pool layout shown is simply a concrete way to discuss the features/issues in the question)

pool
pool/root_dataset
pool/root_dataset/.recycle
pool/root_dataset/.zfs/snapshots
pool/root_dataset/dedupable_data_dataset
pool/root_dataset/dedupable_data_dataset/.recycle
pool/root_dataset/dedupable_data_dataset/.zfs/snapshots
pool/root_dataset/dedupable_data_dataset/user1_files
pool/root_dataset/dedupable_data_dataset/user2_files
pool/root_dataset/dedupable_data_dataset/shared_user_files
pool/root_dataset/dedupable_data_dataset/drivers and installers
pool/root_dataset/non_dedupable_data_dataset
pool/root_dataset/non_dedupable_data_dataset/.recycle
pool/root_dataset/non_dedupable_data_dataset/.zfs/snapshots
pool/root_dataset/non_dedupable_data_dataset/videos
pool/root_dataset/non_dedupable_data_dataset/audio
pool/root_dataset/non_dedupable_data_dataset/compressed_HDD_images ​

Some features of this hypothetical pool:

• Nested datasets: The pool contains a couple of nested datasets. This might be needed any time different ZFS settings apply to different subsets of files in the same "set". In this case let's assume the user's pool, although presented as one share, contains a disjoint mix of highly dedupable files (backups/documents) and highly non-dedupable files (movies/mp3s). It could be due to any other ZFS settings affecting some but not all files, but this is probably simplest for an example Although dedup is pool-wide, it's efficient to only try and dedup a file on writing if it's likely to be dedupable, so it's optimal not even attempting to dedup any video or audio files being saved on the NAS - by using sub-datasets the user ensures ZFS dedups the dedupables, doesn't dedup the movies+audio, and both kinds of file are stored optimally for speed+disk space. (This much is taken from "real life" - on mine I get almost no dedup on movies and 4x dedup for the rest combined: splitting off the non-dedupable data into a separate subdataset instead of just a subdir lets writes to that part of the pool run much faster because it doesn't have to try deduping it when it's almost certain to fail). So we have a dataset that the user sees as a single file share but behind the scenes it's actually a root dataset plus two nested datasets.
• Visible recycler: We want the users to be able to undelete their deleted files, so recycle is active and not hidden.
• Visible snapshots: Provided we make the mountpoint names short enough, it would be nice to let a user explore their own snapshots, by making those visible and accessible across Samba as well (for the times when it's more efficient/easier to explore old versions in a subtree instead of using the "Previous versions/shadow_copy2).
• ACL's generally inherited with a couple of exceptions: Finally, we want to have some limited access dirs, so we use a fairly simple set of allow+deny ACLs combined with Samba's forced inheritance where appropriate. It's minimal: user1_files might have a different set of ACLs from user2_files, but otherwise everything inherits above and below them.

None of these are especially esoteric aims, and all of them are supported by the software. My question is then about the "unexpected" issues an end user might experience, and their mitigation:

1. Exploring snapshots in Windows Explorer: When the user explores the current file systems, it's presented via Samba and any other tool (eg SCP) as a unified file system; the nested dataset aspect isn't apparent. Suppose the user tries to do the same with last week's snapshot (they want to search the tree for an elusive driver). When they explore the tree, I'm guessing they will end up at pool/root_dataset/.zfs/snapshots rather than pool/root_dataset/dedupable_data_dataset/.zfs/snapshots, won't they? Do the symlinks/mounts that make dedupable_data_dataset look like it's a regular subdir of root_dataset still exist as required in the snapshots, or does ZFS create them on demand as it does for other cases? If not, what's the easiest way to not confuse a user who finds that in snapshots, their expected Windows Explorer tree truncates and they have to manually navigate to one of two different trees depending on the file?
2. Undeleting a directory in Samba: a user uses Samba shadow_copy2 / "Previous Versions" to restore a deleted dir in the current dataset. Can they be sure the undelete will actually undelete the entire deleted dir, if that dir contained a nested subdir? It would have to pull relevant files out of .recycle in 2 datasets. Moving them *into* .recycle on 2 datasets was handled transparently by ZFS, but will moving them *out* to undelete be as smooth?
3. ACL handling in snapshots: The same user is exploring their snapshots. But now the ACLs, calculated from the top of the tree, are split across snaps in two datasets. Will they still be evaluated as one would expect for a true snapshot - the same as the ACLs that would have been calculated at the point of snapshotting? Or will there be situations where ACLs on a nested snapshot evaluate to an "unexpected" answer?
4. ACL handling in snapshots if the containing dataset doesn't have concurrent snaps giving ACLs at the same time, or the concurrent snaps were zfs destroy'ed: What happens if the user accidentally destroys the 1 Feb 2018 snapshot of root_dataset, but then realises they need to get a file out of the 1 Feb 2018 snapshot of /.../videos? The OS will presumably need to evaluate the ACLs on that dir, to do which it needs to get the relevant ACLs for root_dataset/.zfs/snapshot/auto@2018-02-01..../ etc where it is nested... but any ACLs stored with that snapshot don't seem to exist any longer. That could mean there's no access at all (even root can't help because ACLs+owner info on the snapshot is read only even to root), or it could mean anyone can access them and security breaks (if the necessary snaps from the containing dataset don't exist), or that the wrong ACLs are applied (if the containing dataset's ACLs changed since 1 Feb 2018). What happens, and how is security handled?

I'm sure there are other situations which could give anomalies. But these will do as working examples to focus an answer. My aim in asking is to understand how ACLs and "transparent" nesting of datasets, work, when the focus moves to snapshots of those datasets for any reason.

Are the necessary mounts created/handled so that the dataset still appear to be a single structure? Are necessary/contemporaneous ACLs managed? Will file searching still work across the NAS? what if a snap exists for a subdataset, how does it ensure it uses the right ACLs from a snap of the containing dataset and what if such a snap doesn't seem to exist?

As this is a general request for understanding and "best mitigation possible", it would help to have detailed and comprehensive replies if anyone happens to know the detailed answers!

Thank you for any help and insight you're able to give!

 

[dlavigne]

Did you find the answers to any of these questions?

 

[Stilez]

No. I'm attempting workarounds and exploration but it could take some weeks.