One of the more beneficial features of the ZFS filesystem is the way it allows for tiered caching of data through the use of memory, read and write caches. By optimizing memory in conjunction with high speed SSD drives, significant performance gains can be achieved for your storage. The first level of caching in ZFS is the Adaptive Replacement Cache (ARC), which is composed of your system’s DRAM. It is the first destination for all data written to a ZFS pool, and it is the fastest (i.e. lowest-latency) source for data read from a ZFS pool. When data is requested from ZFS, it looks first to the ARC; if it is there, it can be retrieved extremely quickly (typically in nanoseconds) and provided back to the application.
This provides greater read performance improvements by orders of magnitude over older methodologies like short-stroked spinning disks, which are power hungry and expensive.
The contents of the ARC are balanced between the most recently used (MRU) and most frequently used (MFU) data. This balance is important. Normally, running a backup product would walk the entire file system and effectively invalidate the cache. Since ZFS utilizes algorithms to track frequently used data in addition to recently used data, your cache devices will still provide performance improvements after a backup.
ZFS brings frequently and recently used data to the highest performing storage, first to system memory, then to caching devices, allowing for flash media performance without the cost.
Level 2 Adaptive Replacement Cache (L2ARC)
Once all the space in the ARC is utilized, ZFS places the most recently and frequently used data into the Level 2 Adaptive Replacement Cache (L2ARC). The L2ARC is usually larger than the ARC so it caches much larger datasets. ZFS will accelerate random read performance on datasets far in excess of the size of the system main memory, which avoids reading from slower spinning disks as much as possible.
The ZFS Intent Log (ZIL)
ZFS commits synchronous writes to the ZFS Intent Log, or ZIL. iXsystems offers super-capacitor-backed DRAM as ZIL devices, which allows cached writes to be committed to nonvolatile storage so that they’re protected in the event of a sudden power failure. This allows synchronous writes to be made at the speed of the ZIL, accelerating NFS performance.
Determining Cache and Pool Size
After the ARC, ZIL and L2ARC comes the hard disks, comprising the ZFS pool. This tier is where your data lives and is usually composed of high capacity hard disks. Performance at this tier is the lowest of all, as it depends on spinning disks rather than flash drives.
IOPS by Type of Storage
In order to configure your FreeNAS or TrueNAS system for ideal performance between cache and pool, it is important to determine the Working Set Size of your system. Knowing the active data and performance requirements of your storage environment will allow you to put together a system that maximizes performance. Some questions that can help determine the Working Set Size are as follows:
- What percentage of your total data is “active?” (20% is not unusual)
- How will the remaining data be dealt with?
- One file or a set of data accessed simultaneously?
- How many users or applications?
- How many people will log in simultaneously?
- What is the average file size and how many?
- What is your workflow?
- What percentage of your usage is read versus write?
Once the Working Set Size is determined, one can select the optimal drive to maximize performance. In regard to types of SSD cache drives, the L2ARC read cache does not require as high performance, as the data is already stored on disk and there is no risk of data loss. The ZIL requires higher quality storage devices/memory, as the data has not made it to the storage pool yet. In the event of a power loss, cheaper flash memory (MLC flash) can lose write data. SLC Flash Memory devices, on the other hand, do not have this issue.
Conclusion
ZFS Caching can be an excellent way to maximize your system performance and give you flash speed with spinning disk capacity. TrueNAS capitalizes on this technology and the staff at iXsystems have the expertise to help you design a system that fits your needs and leverages the caching capabilities of ZFS to their full extent. For more information on TrueNAS, visit ixsystems.com/truenas or call 1-855-GREP-4-IX.