opportunity to learn something new today
Perfect! Once I get that out of the way, I can go back to hitting things with hammers.
How well does the standalone
lz4
executable correlate to ZFS behavior?I thought these both seemed relevant:
A simple (real world) ZFS compression speed an compression ratio benchmark
Posted in r/zfs by u/Schmidsfeld • 166 points and 37 comments
old.reddit.com
OpenZFS: Understanding Transparent Compression
OpenZFS’ transparent compression is one of the many compelling yet misunderstood features. Read about more about its amazing potential performance benefits in our “OpenZFS in Depth” series.
klarasystems.com
This has a bunch of meaty refererences:
LZ4 vs. ZStd
I'm about to create a new pool (TrueNAS-12.0-U1). There doesn't seem to be much info about the different choices and which one to pick. The data is a mixed bag of everything. Small files, big files, source code, incompressible files, VMs, … LZ4 or ZStd? How to decide?
www.truenas.com
ZSTD Benchmarks
fio SSD ZSTD Relative Compression Performance System: Intel Xeon E5-2630 v4 @ 2.20GHz (10 cores, 20 threads), 32GB RAM, 4x 500 Samsing 860 EVO SSDs ZFS config: ashift=12 atime=off recordsize=128k arc_max=256m Workload: fio uncompressible data to SSD: bs=128k rw=randwrite numjobs=8 iodepth=16 siz...
docs.google.com
=lz4
achieved 98% the speed of =off
.So my thoughts are a combination of these factors -
A 2% worst-case penalty isn't significant
Any real-world mix of data will compress much more than 2%
Systems with spinning disks benefit greatly from avoiding IOPS
I would like to see a real-world example of
=off
being faster. I hypothesize that would be an unusually slow CPU, paired with disproportionally fast storage, writing incompressible data.