ZFS Layout questions (or, let me ask this another way)?

[david kennedy]

OK, no hits on my original question so perhaps I was unclear so let me try this from another angle.

I have a SUN X4500 i am looking to convert from Solaris 11.1 to FreeNas 9.

On this box i have the following structure.

z01/home/A
z01/home/B
z01/home/C
z01/home/D
z01/home/E
z01/Share
z01/Share/A
z01/Share/B
etc

Now when i created a similar structure in FreeNas (using the gui) i found a few oddities.

1) lets say i put Share into a jail (mounted on /media). The jail can see "A", "B" etc.

If i make a folder under /media/Share/A/MyNewFolder this is NOT visible under the non-jail account but is visible on the NFS share (using the real ip, not the jails). If i set the NFS permissions correctly i can create files and folders under /media/Share/A/ and they are visible in the jail, but not the "host".

2) the storage accounting also seems weird. items stored in /media/Share/A/MyNewFolder are reported under /media/Share/ and not /media/Share/A which shows like 127 bytes or such.

As I am totally new to FreeNas i just need a bit of guidance on the best practices on creating / sharing ZFS filesystems under Freenas. For what it is worth the file systems are shared out over NFS under solaris and work exactly as one would expect but only because they are shared out at the "child" level, IE:

MyBox:/z01/Share/A on /mnt/ShareA
MyBox:/z01/Share/B on /mnt/ShareB
etc

Instead of sharing MyBox:/z01/Share to /mnt/Share which i think is causing the issue (nested sharing in Solaris isn't really supported).

So, should I create file systems like :
z01/Share/A
z01/Share/B

But mount them as /media/A and /media/B instead?


Hopefully this is more clear and someone can give me a hand. The X4500 is a solid machine, but solaris is a little heavy for what it needs to do (NAS).

 

[david kennedy]

I just created a VM and so i can give specifics


On the client pc
df -h
192.168.1.203:/mnt/z01/Share 8.4G 192K 8.4G 1% /mnt/a

cd /mnt/a
ls
A
B
ls /mnt/a/A
def
ghi

So this is good, the client sees the same folders that i created as NFS file systems.
Now, lets see what the default "shell" shows

zfs list
z01/Share /mnt/z01/Share
z01/Share/A /mnt/z01/Share/A
z01/Share/B /mnt/z01/Share/B

We agree.
Now lets go to the file system

cd /mnt/z01/Share/A
ls
abcdef ( a test file i created via "touch").

Lets try from the jail
root@MediaServices:/ # cd /media/share/A
# ls
def
ghi

I created def IN the jail, and ghi via the NFS mount on another PC
Where is "abcdef"? Its a file visible under the default 'shell' (outside the jail) but not visible via NFS or inside the jail.

Similarly "def" and "ghi" are visible inside the jail and via nfs but not on the default "shell"?

Now via another PC and the NFS mount lets make a file:
dd if=/dev/zero of=file.txt count=10000 bs=1024

and sure enough its there and under the user i have mapped NFS to.

Back to the root system (outside the jail)
zfs list
z01/Share/ 10.6M 8.35G 10.1M /mnt/z01/Share
z01/Share/A 144K 8.35G 144K /mnt/z01/Share/A

Notice the 10MB is not allocated to z01/Share/A even though that is where the file is?
Yet the JAIL agrees that the 10MB is in "A":
cd /media
du -h

9.9M ./A (folder A has 10MB of data)
8.5k ./B (folder B has 0MB of data)
10M . The total is correct.

 

[david kennedy]

OK, ready to answer my own question here.

FreeNas does have the same issue as solaris with nested shares.

One of the big differences is how they behave.

What threw me off is that when i share out /mnt/share/ and it has children i didnt really expect them to be visible and also unusable?

So it looks like what is happening is /mnt/share/A is actually a directory under share, and not the "A" file system.

I will still create the "tree" as i like how zfs "rolls up" the storage but share and mount them as seperate file systems.