ZFS ARC metadata size minimum

dcm39

Cadet
Joined
Mar 13, 2019
Messages
2
I have a large 100s TB ZFS system that is suffering from slow read performance - it has 768GB RAM, but only uses a tiny fraction of that for metadata (<5%).

With ZFS on linux on another very similar system I'm able to double the read performance by increasing the "zfs_arc_meta_min" module parameter. This forces the arc to cache a LOT more metadata (>10x) and the performance boost is huge for our workflow which consists mainly of sequential read.

As far as I can tell there is no "arc_meta_min" tunable on freenas. Is there a similar way force freenas zfs to cache more metadata (11.1-U5)?
 

mav@

iXsystems
iXsystems
Joined
Sep 29, 2011
Messages
1,428
There is vfs.zfs.arc_meta_min loader tunable on FreeNAS. It is not present as sysctl, but should work if you set it as loader tunble in WebUI.
 

scurrier

Patron
Joined
Jan 2, 2014
Messages
297
@dcm39 I am having sequential read performance issues on a much smaller system (3x 4tb striped mirrors, 32gb ram) that I suspect arc metadata may play a part. Can you report back on your results of tuning vfs.zfs.arc_meta_min and anything else you might have found? A few sentences would go a long way.
 

dcm39

Cadet
Joined
Mar 13, 2019
Messages
2
After a lot of struggling, we eventually gave up and moved the system to CentOS and saw a massive boost in performance. Part of this we attribute to being able to successfully increase the arc_meta_min. Part we attribute to the larger max blocksize for NFS on linux (1M).
 

scurrier

Patron
Joined
Jan 2, 2014
Messages
297
Interesting. Thanks for replying.
 

mav@

iXsystems
iXsystems
Joined
Sep 29, 2011
Messages
1,428
Already after 11.2-U5 release there was a fix for ARC MRU/MFU balance, that in some scenatios could lead to very small MFU size and low performance. In comimng soon 11.2-U6 ARC should better adapt to different workloads.
 

scurrier

Patron
Joined
Jan 2, 2014
Messages
297
@mav@ is metadata similarly subject to mfu/mfu policies as normal file data is?
 
Top