Special Metadrive

[kspare]

Running Truenas 12.0u5, I have about 4tb of data (all terminal servers on nfs share)

I left the special meta drive block size at zero.

My understanding is that this will just write the meta data for the volume there and read it to access data. Keep in mind we have 1tb of ram and 12tb of l2arc.

watching iostat I do see write operations to the special drive, but I *never* see reads.

left at the default of zero, what exactly is being written there?

Block Size Histogram

block psize lsize asize
size Count Size Cum. Count Size Cum. Count Size Cum.
512: 1.22K 626K 626K 123 61.5K 61.5K 0 0 0
1K: 2.31K 2.40M 3.01M 87 99.5K 161K 28 41.5K 41.5K
2K: 34.9K 116M 119M 52 133K 294K 8 22K 63.5K
4K: 613K 2.39G 2.51G 35.1K 140M 141M 608K 2.38G 2.38G
8K: 928K 9.34G 11.9G 17 202K 141M 856K 8.49G 10.9G
16K: 2.31M 51.4G 63.3G 3.59K 57.5M 198M 2.36M 53.1G 64.0G
32K: 6.07M 284G 347G 180K 5.62G 5.81G 6.13M 286G 350G
64K: 14.6M 1.28T 1.61T 4 352K 5.81G 14.6M 1.28T 1.62T
128K: 19.0M 2.37T 3.99T 43.3M 5.41T 5.41T 19.0M 2.37T 3.99T
256K: 0 0 3.99T 0 0 5.41T 3.82K 1.16G 3.99T

looking at this, it looks like if I set the block size to 16kb it would put some data on the meta drive if I moved all my data off and on again (easy for me to do)

So...long story short, to gain some performance/use the special metadrive properly, do I leave it at zero, or do I change it to 16kb?

 

[QonoS]

Zero mains only metadata will be written to metadata drive. 16kb means any data that fits into a 16kb block will be written to the metadata drive instead of the data drives.

Regarding that "only writes" phenomenon: Option "primarycache=all" - which is default - results in the metadata being kept in RAM and since you have 1TB surely all reads will done from RAM since its faster. Try setting "primarycache=none" and watch out for metadata reads again.