Bluefin Recommended Settings and Optimizations

[anodos]

Apple SMB2/3 protocol extensions means vfs_fruit, doesn't it?!

vfs_fruit

Do you have a Mac? Do you use tags and labels for files (these are stored in the Xattrs)?

Yes, I've very aware of these things. Xattrs / streams work without vfs_fruit.

 

[Daisuke]

I’m going to keep the setting enabled into guide, for simplicity.

 

[crk1918]

That's the correct info, start from Edit ACLs you will be able to define there the second user and related permissions. Keep in mind I never used multiple users through POSIX ACLs, can you please share your experiences here, for others?

I seem to have found my answer: YouTube Video by Sauber-Lab UK, I guess the most important thing is to understand the "mask", which I found this comment by @haydenstith from the YouTube platform: "Mask sets the maximum permissions available to everyone. If the mask is set to just read then everyone with read permissions will be able to read as intended but at the same time everyone with writes or execute privileges will only be able to read the file as well."


Also, after some searching, find user anodos pointed out from other posts about this kind of issue: explanation 1 and explanation 2

More Info:
https://wpollock.com/AUnix1/FilePermissions.htm
https://tylersguides.com/guides/linux-acl-permissions-tutorial/


I hope these statements are correct and I will test them again. If you have more accurate information, you can tell me here.

I just would like the official documentation to make it clearer to us.

 

[anodos]

That will change the default POSIX to NFS/SMB, which is exactly what I want to avoid. Please let me know if I’m correct.

Speaking in broad terms, our recommended defaults are to use the SMB dataset preset for SMB shares (which uses NFSv4-style ACLs). What is the specific problem that you encountered that requires moving away from the default?

 

[Daisuke]

What is the specific problem that you encountered that requires moving away from the default?

The default ACL type is POSIX, when you create a dataset:



This default setting never generated any permission issues, while using the SMB share in Linux, Mac or Windows. From your documentation:

Changes to ACL type affect how on-disk ZFS ACL is written and read. When the ACL type is changed from POSIX to NFSv4, no migration is performed for default and access ACLs encoded in the posix1e acl extended attributes to native ZFS ACLs. When ACL type is changed from NFSv4 to POSIX, native ZFS ACLs are not converted to posix1e extended attributes, but the native ACL will be used internally by ZFS for access checks. This means that the user must manually set new ACLs recursively on the dataset after ACL type changes in order to avoid unexpected permissions behavior. This action will be destructive, and so it is advised to take a ZFS snapshot of the dataset prior to ACL type changes and permissions modifications.

I don't see the need to change the ACL type from POSIX to SMB/NFSv4 and go through all these manual tasks which also introduce destructive actions requiring ZFS snapshots, when default POSIX setting works as intended.

 

[bcat]

AFAICT, those docs cover changing acltype in existing datasets. If POSIX ACLs are working well for your setup, then I assume there's no reason to change them. However, as I've mentioned elsewhere, the official SCALE docs on creating SMB shares recommend the "SMB" share type (including NFSv4 ACLs) for new datasets intended to be shared over SMB.

And this makes sense, since NFSv4 ACLs align well with Windows ACLs, whereas POSIX ACLs do not. (Indeed, this is one of the major reasons I ended up with TrueNAS in the first place: It lets me host Samba shares that actually play nice with Windows permissions, since it supports NFSv4 ACLs at the filesystem level rather than papering over POSIX ACLs.)

So while I'm sure there are some use cases where SMB shares with POSIX ACLs make the most sense, I don't think iXsystems intended to promote that as the default or recommended config. (In fact, their docs appear to say the opposite.)

 

[Daisuke]

The official SCALE docs on creating SMB shares recommend the "SMB" share type (including NFSv4 ACLs) for new datasets intended to be shared over SMB.

That’s exactly my point, I’m waiting on an answer from @anodos related to that specific requirement. The explanation provided into documentation is not sufficient, especially that POSIX works in Windows. I’m reiterating what I mentioned in an earlier post, is the end-user who decides how to use the software, not the software provider.

If there are valid requirements for specific cases, then the end-user can decide if additional change actions are required, based on clear documentation.

 

[bcat]

That makes sense. There are more details on the choice between NFSv4 ACLs and POSIX ACLs for SMB shares in the ACL primer. Does that answer your questions?

 

[Daisuke]

Does that answer your questions?

It does, thank you. I will add this information into guide, so users can decide the best action course for their setup. On my side, I’ll continue using POSIX.

 

[oblivioncth]

Not trying to be an a-hole about it, just sharing info for awareness:

SMB Shares
-snip-

SMB Shares Configuration
-snip-

To avoid any hostPath conflicts, set your SMB share to /mnt/software/opt and your application hostPath to /mnt/software/opt/downloads. Assigning an SMB share directly to /mnt/software pool is not considered good practice.

As previously speculated, this no longer works as of 22.12.1.

 

[Belperite]

As previously speculated, this no longer works as of 22.12.1.

Yes I've just run into this. Sigh.

 

[Master-DXT]

Yes I've just run into this. Sigh.

Same here, lol.

 

[Daisuke]

As previously speculated, this no longer works as of 22.12.1

I'm going to update the guide soon with additional NFS shares.

 

[manueltts]

Hi,

Many thanks for these tips. Regarding `recordsize`, here is a long one-liner that provides more granular information on the file size distribution (with percentiles and percentages per size range, and cumulative percentages besides the average and such):

Code:

find /mnt/apps/opt/downloads -type f -printf '%s\n' | awk '{s+=$1} {n++; a[NR - 1] = $1} END {asort(a); m = a[1]/1024; M = a[NR]/1024; q50 = a[int(n*0.5)]/1024; q75 = a[int(n*0.75)]/1024; q90 = a[int(n*0.9)]/1024; q95 = a[int(n*0.95)]/1024; q975 = a[int(n*0.975)]/1024; q99 = a[int(n*0.99)]/1024; printf "Count: %d\nAverage: %.2fK\nMin: %.2fK\nMax: %.2fK\n50th percentile: %.2fK\n75th percentile: %.2fK\n90th percentile: %.2fK\n95th percentile: %.2fK\n97.5th percentile: %.2fK\n99th percentile: %.2fK\n", n, s/n/1024, m, M, q50, q75, q90, q95, q975, q99; c4=0; c8=0; c16=0; c32=0; c64=0; c128=0; c526=0; c1m=0; for (i=1;i<=n;i++) {if (a<=4096) {c4++} else if (a<=8192) {c8++} else if (a<=16384) {c16++} else if (a<=32768) {c32++} else if (a<=65536) {c64++} else if (a<=131072) {c128++} else if (a<=526336) {c526++} else if (a<=1048576) {c1m++}}; printf "Percentage of files below 4K: %.2f%%\n", c4/n*100; printf "Percentage of files between 4K and 8K: %.2f%% (%.2f%%)\n", c8/n*100, (c4+c8)/n*100; printf "Percentage of files between 8K and 16K: %.2f%% (%.2f%%)\n", c16/n*100, (c4+c8+c16)/n*100; printf "Percentage of files between 16K and 32K: %.2f%% (%.2f%%)\n", c32/n*100, (c4+c8+c16+c32)/n*100; printf "Percentage of files between 32K and 64K: %.2f%% (%.2f%%)\n", c64/n*100, (c4+c8+c16+c32+c64)/n*100; printf "Percentage of files between 64K and 128K: %.2f%% (%.2f%%)\n", c128/n*100, (c4+c8+c16+c32+c64+c128)/n*100; printf "Percentage of files between 128K and 526K: %.2f%% (%.2f%%)\n", c526/n*100, (c4+c8+c16+c32+c64+c128+c526)/n*100; printf "Percentage of files between 526K and 1M: %.2f%% (%.2f%%)\n", c1m/n*100, (c4+c8+c16+c32+c64+c128+c526+c1m)/n*100; printf "Percentage of files over 1M: %.2f%% (%.2f%%)\n", (n - c526 - c1m - c128 - c64 - c32 - c16 - c8 - c4)/n*100, 100.00}' 

Cheers!

 

[WolfpactVI]

Not trying to be an a-hole about it, just sharing info for awareness:


As previously speculated, this no longer works as of 22.12.1.

So I still have a question about the whole SMB access thing. The only app I have at the moment is Plex. I have one pool (datapool1) and below that sits various datasets. One is the ix-applications, on is caslled "media", one called "photos", etc. Media is also an SMB share so I could transfer all of my movies and shows off my PC and then point Plex to it and turn it into the Plex library. As expected, once I updated to Bluefin, Plex does not start because of the host path safety check.

My first question has to do with Daisuke's guide - it seems like it's saying that you can have a parent dataset as an SMB share but then point an app's host path to a child dataset that is not explicitly an SMB share? That is seems.....counterintuitive.....to me. Could you explain how/why this works? Will I still be able to access said child dataset via SMB through the parent?

But now it seems like oblivioncth is saying even this does not work in 22.12.1 - is that correct? So then what IS the correct way to allow an app to access an SMB share? I'm getting ready to set up Nextcloud, and I want to it to be able to access my "photos" dataset. But I still want to be able to transfer photos from my PC or laptop or phone into that dataset via SMB. Or is this basically impossible in Bluefin without turning off the host path safety check? I realize there is good reason they added it in, and I'd like to leave it on and learn the correct way to set things up.

Thanks for any suggestions or clarifications you have!

 

[oblivioncth]

My first question has to do with Daisuke's guide - it seems like it's saying that you can have a parent dataset as an SMB share but then point an app's host path to a child dataset that is not explicitly an SMB share? That is seems.....counterintuitive.....to me. Could you explain how/why this works? Will I still be able to access said child dataset via SMB through the parent?

This setup still fundamentally exhibited the same issue, it only worked because of an oversight with how the path check was written in 22.12.0.

But I still want to be able to transfer photos from my PC or laptop or phone into that dataset via SMB. Or is this basically impossible in Bluefin without turning off the host path safety check? I realize there is good reason they added it in, and I'd like to leave it on and learn the correct way to set things up.My first question has to do with Daisuke's guide - it seems like it's saying that you can have a parent dataset as an SMB share but then point an app's host path to a child dataset that is not explicitly an SMB share? That is seems.....counterintuitive.....to me. Could you explain how/why this works? Will I still be able to access said child dataset via SMB through the parent?

These are two separate posts in the same thread

Recommended Plex workflow with sharing when accounting for HostPath validation?

First post here, so excuse the probably-obvious question, but are all these workarounds in these various threads just intended to keep things within the Applications UI? Because it does seem - as another poster commented on another of these similar threads - that the plane is being built while...

www.truenas.com

Recommended Plex workflow with sharing when accounting for HostPath validation?

First post here, so excuse the probably-obvious question, but are all these workarounds in these various threads just intended to keep things within the Applications UI? Because it does seem - as another poster commented on another of these similar threads - that the plane is being built while...

www.truenas.com

tl;dr - Use TrueCharts containers and setup an NFS share between the container and the same directory you share via SMB. This is what Daisuke will end up recommending in the revised version of the guide and what should work for most.

Alternatively you can go wild west and disable host path validation, but as the first link explains its highly recommended to avoid doing that until the "better way" for doing so is hopefully introduced in 23.10, by which point it will be a reasonable more advanced alternative.

 

[WolfpactVI]

This setup still fundamentally exhibited the same issue, it only worked because of an oversight with how the path check was written in 22.12.0.


These are two separate posts in the same thread

Recommended Plex workflow with sharing when accounting for HostPath validation?

First post here, so excuse the probably-obvious question, but are all these workarounds in these various threads just intended to keep things within the Applications UI? Because it does seem - as another poster commented on another of these similar threads - that the plane is being built while...

www.truenas.com

Recommended Plex workflow with sharing when accounting for HostPath validation?

First post here, so excuse the probably-obvious question, but are all these workarounds in these various threads just intended to keep things within the Applications UI? Because it does seem - as another poster commented on another of these similar threads - that the plane is being built while...

www.truenas.com

tl;dr - Use TrueCharts containers and setup an NFS share between the container and the same directory you share via SMB. This is what Daisuke will end up recommending in the revised version of the guide and what should work for most.

Alternatively you can go wild west and disable host path validation, but as the first link explains its highly recommended to avoid doing that until the "better way" for doing so is hopefully introduced in 23.10, by which point it will be a reasonable more advanced alternative.

NFS will only work in Truecharts apps, not the official apps? If the Truecharts apps are so much better (I assume that's what you meant by "Using the charts for your services is much more efficient and hands free once you get it setup"), then why do both exist? But I guess that's getting off topic. Regardless, my current Plex app that carried over from Angelfish is an official app, so the NFS solution will not work in this case, correct? And is it just me, or does this seem to add an extra layer of complexity to something that used to be (was supposed to be?) fairly straightforward to set up?

 

[oblivioncth]

NFS will only work in Truecharts apps, not the official apps? If the Truecharts apps are so much better (I assume that's what you meant by "Using the charts for your services is much more efficient and hands free once you get it setup"), then why do both exist? But I guess that's getting off topic. Regardless, my current Plex app that carried over from Angelfish is an official app, so the NFS solution will not work in this case, correct? And is it just me, or does this seem to add an extra layer of complexity to something that used to be (was supposed to be?) fairly straightforward to set up?

Yes only for them. I do think they tend to be more flexible/capable, but it's because the entire mission of TrueCharts is implementing... well, charts. The official ones by iXsystems are only a portion of what they can afford to focus on so they're clearly favoring stability and adding new charts at a slower pace. I would say that typically, most people in the homelab setting will want to use the TrueChart variants of charts and that the official ones are better suited for more business oriented environments. I am a bit puzzled what the official guidance is at this point though for users in this situation though given that they only really have the option of disabling HostPath validation or temporarily killing charts and spinning up network shares when they want to transfer data.

One of their own (unrelated to this issue) announcements referenced using NFS shares via a TrueCharts app so I wouldn't be surprised if they added NFS shared to their charts in the future as well. The core functionality is built into Kubernetes.

At the very least I think this awkward situation is what prompted them to start working on the more granular HostPath setting for 23.10.

So in your particular situation, outside of coming up with some really creative wacky stuff, you either need to just rough it until 23.10 or disable HostPath validation until then. If you're careful with what you put on your system, especially if you don't alter it that much once it's configured and just let it run as an appliance then doing so might not be a big deal for you during that timespan. One of the head engineers of iXsystems has stated they've done this in the past themselves, though it goes without saying that they're very aware of the risks.

I suppose you could also migrate to the TrueCharts version of Plex, which likely wouldn't be that hard if you know how to move the data and config over, but I can understand if you'd like to avoid that.

 

[WolfpactVI]

Yes only for them. I do think they tend to be more flexible/capable, but it's because the entire mission of TrueCharts is implementing... well, charts. The official ones by iXsystems are only a portion of what they can afford to focus on so they're clearly favoring stability and adding new charts at a slower pace. I would say that typically, most people in the homelab setting will want to use the TrueChart variants of charts and that the official ones are better suited for more business oriented environments. I am a bit puzzled what the official guidance is at this point though for users in this situation though given that they only really have the option of disabling HostPath validation or temporarily killing charts and spinning up network shares when they want to transfer data.

One of their own (unrelated to this issue) announcements referenced using NFS shares via a TrueCharts app so I wouldn't be surprised if they added NFS shared to their charts in the future as well. The core functionality is built into Kubernetes.

At the very least I think this awkward situation is what prompted them to start working on the more granular HostPath setting for 23.10.

So in your particular situation, outside of coming up with some really creative wacky stuff, you either need to just rough it until 23.10 or disable HostPath validation until then. If you're careful with what you put on your system, especially if you don't alter it that much once it's configured and just let it run as an appliance then doing so might not be a big deal for you during that timespan. One of the head engineers of iXsystems has stated they've done this in the past themselves, though it goes without saying that they're very aware of the risks.

I suppose you could also migrate to the TrueCharts version of Plex, which likely wouldn't be that hard if you know how to move the data and config over, but I can understand if you'd like to avoid that.

Thanks for all of your help clarifying this. I'll have to weigh my options. I may give the TrueCharts versions a whirl. Could you point me to a good guide for installing a Truecharts app that includes the internal NFS share setup, if such a thing exists?

 

[imvi]

They have an article on this topic here