I hope you can help me understand what's going on here with my original dataset using lz4 and the replicated one using gzip-9.
Two FreeNAS servers, 1# and #2, both has a new volume with nothing special going on.
The dataset in question that is being replicated has all kinds of different data stored.
Server #1:
Dataset 'nas' contains 7TB used data. Has default settings storage wise, which also means lz4 compression. The compression ratio is 1.02x. Contains only one snapshot that was replicated to server #2.
Server #2:
Replicated and untouched readonly dataset 'nas' contains 7.6TB used data. Has default settings storage wise, except using gzip-9. The compression ratio is 1.02x. Contains only one snapshot that was the replicated one.
In summary:
Dataset on server #1: 7TB used, lz4, 1.02x compression rate
Dataset on server #2: 7.6TB used, gzip-9, 1.02x compression rate
Note that both datasets reports "logicalreferenced 7.17T".
Question:
The dataset on server #2 is about 0.7TB bigger, even though it probably should be more compressed given gzip-9, but both datasets reports the same compressratio of 1.02x so it seems that gzip-9 didn't compress the data more than lz4.
However, when both datasets reports the same compression ratio, why is the replicated dataset 0.7TB bigger?
Results of zfs get all:
Server 1#:
Server #2:
Two FreeNAS servers, 1# and #2, both has a new volume with nothing special going on.
The dataset in question that is being replicated has all kinds of different data stored.
Server #1:
Dataset 'nas' contains 7TB used data. Has default settings storage wise, which also means lz4 compression. The compression ratio is 1.02x. Contains only one snapshot that was replicated to server #2.
Server #2:
Replicated and untouched readonly dataset 'nas' contains 7.6TB used data. Has default settings storage wise, except using gzip-9. The compression ratio is 1.02x. Contains only one snapshot that was the replicated one.
In summary:
Dataset on server #1: 7TB used, lz4, 1.02x compression rate
Dataset on server #2: 7.6TB used, gzip-9, 1.02x compression rate
Note that both datasets reports "logicalreferenced 7.17T".
Question:
The dataset on server #2 is about 0.7TB bigger, even though it probably should be more compressed given gzip-9, but both datasets reports the same compressratio of 1.02x so it seems that gzip-9 didn't compress the data more than lz4.
However, when both datasets reports the same compression ratio, why is the replicated dataset 0.7TB bigger?
Results of zfs get all:
Server 1#:
Code:
tank/ds/nas type filesystem - tank/ds/nas creation Thu Oct 24 0:33 20 19 - tank/ds/nas used 7.02T - tank/ds/nas available 8.28T - tank/ds/nas referenced 7.02T - tank/ds/nas compressratio 1.02x - tank/ds/nas mounted no - tank/ds/nas quota none default tank/ds/nas reservation none default tank/ds/nas recordsize 128K default tank/ds/nas mountpoint /mnt/tank/ds/ replication/nas default tank/ds/nas sharenfs off default tank/ds/nas checksum on default tank/ds/nas compression lz4 inherited from tank tank/ds/nas atime off inherited from tank tank/ds/nas devices on default tank/ds/nas exec on default tank/ds/nas setuid on default tank/ds/nas readonly on local tank/ds/nas jailed off default tank/ds/nas snapdir hidden default tank/ds/nas aclmode restricted inherited from tank/ds tank/ds/nas aclinherit passthrough inherited from tank tank/ds/nas canmount on default tank/ds/nas xattr on default tank/ds/nas copies 1 default tank/ds/nas version 5 - tank/ds/nas utf8only off - tank/ds/nas normalization none - tank/ds/nas casesensitivity insensitive - tank/ds/nas vscan off default tank/ds/nas nbmand off default tank/ds/nas sharesmb off default tank/ds/nas refquota none default tank/ds/nas refreservation none default tank/ds/nas primarycache all default tank/ds/nas secondarycache all default tank/ds/nas usedbysnapshots 0 - tank/ds/nas usedbydataset 7.02T - tank/ds/nas usedbychildren 0 - tank/ds/nas usedbyrefreservation 0 - tank/ds/nas logbias latency default tank/ds/nas dedup off default tank/ds/nas mlslabel - tank/ds/nas sync standard default tank/ds/nas refcompressratio 1.02x - tank/ds/nas written 0 - tank/ds/nas logicalused 7.17T - tank/ds/nas logicalreferenced 7.17T - tank/ds/nas volmode default default tank/ds/nas filesystem_limit none default tank/ds/nas snapshot_limit none default tank/ds/nas filesystem_count none default tank/ds/nas snapshot_count none default tank/ds/nas redundant_metadata all default tank/ds/nas org.freenas:description Original nas inherited from tank/ds tank/ds/nas org.freebsd.ioc:active yes inherited from tank tank/ds/nas@auto-20191023.2031-100y type snapshot - tank/ds/nas@auto-20191023.2031-100y creation Wed Oct 23 20:31 20 19 - tank/ds/nas@auto-20191023.2031-100y used 0 - tank/ds/nas@auto-20191023.2031-100y referenced 7.02T - tank/ds/nas@auto-20191023.2031-100y compressratio 1.02x - tank/ds/nas@auto-20191023.2031-100y devices on default tank/ds/nas@auto-20191023.2031-100y exec on default tank/ds/nas@auto-20191023.2031-100y setuid on default tank/ds/nas@auto-20191023.2031-100y xattr on default tank/ds/nas@auto-20191023.2031-100y version 5 - tank/ds/nas@auto-20191023.2031-100y utf8only off - tank/ds/nas@auto-20191023.2031-100y normalization none - tank/ds/nas@auto-20191023.2031-100y casesensitivity insensitive - tank/ds/nas@auto-20191023.2031-100y nbmand off default tank/ds/nas@auto-20191023.2031-100y primarycache all default tank/ds/nas@auto-20191023.2031-100y secondarycache all default tank/ds/nas@auto-20191023.2031-100y defer_destroy off - tank/ds/nas@auto-20191023.2031-100y userrefs 0 - tank/ds/nas@auto-20191023.2031-100y mlslabel - tank/ds/nas@auto-20191023.2031-100y refcompressratio 1.02x - tank/ds/nas@auto-20191023.2031-100y written 7.02T - tank/ds/nas@auto-20191023.2031-100y clones - tank/ds/nas@auto-20191023.2031-100y logicalreferenced 7.17T - tank/ds/nas@auto-20191023.2031-100y volmode default default tank/ds/nas@auto-20191023.2031-100y org.freenas:description nas inherited from tank/ds tank/ds/nas@auto-20191023.2031-100y org.freebsd.ioc:active yes inherited from tank
Server #2:
Code:
tank/archive/nas type filesystem - tank/archive/nas creation Sat Oct 26 23:48 2019 - tank/archive/nas used 7.64T - tank/archive/nas available 5.15T - tank/archive/nas referenced 7.64T - tank/archive/nas compressratio 1.02x - tank/archive/nas mounted no - tank/archive/nas quota none default tank/archive/nas reservation none default tank/archive/nas recordsize 128K default tank/archive/nas mountpoint /mnt/tank/archive/nas default tank/archive/nas sharenfs off default tank/archive/nas checksum on default tank/archive/nas compression gzip-9 inherited from tank/archive tank/archive/nas atime off inherited from tank tank/archive/nas devices on default tank/archive/nas exec on default tank/archive/nas setuid on default tank/archive/nas readonly on local tank/archive/nas jailed off default tank/archive/nas snapdir hidden default tank/archive/nas aclmode restricted inherited from tank/archive tank/archive/nas aclinherit passthrough inherited from tank tank/archive/nas canmount on default tank/archive/nas xattr on default tank/archive/nas copies 1 inherited from tank tank/archive/nas version 5 - tank/archive/nas utf8only off - tank/archive/nas normalization none - tank/archive/nas casesensitivity insensitive - tank/archive/nas vscan off default tank/archive/nas nbmand off default tank/archive/nas sharesmb off default tank/archive/nas refquota none default tank/archive/nas refreservation none default tank/archive/nas primarycache all default tank/archive/nas secondarycache all default tank/archive/nas usedbysnapshots 0 - tank/archive/nas usedbydataset 7.64T - tank/archive/nas usedbychildren 0 - tank/archive/nas usedbyrefreservation 0 - tank/archive/nas logbias latency default tank/archive/nas dedup off default tank/archive/nas mlslabel - tank/archive/nas sync standard default tank/archive/nas refcompressratio 1.02x - tank/archive/nas written 0 - tank/archive/nas logicalused 7.17T - tank/archive/nas logicalreferenced 7.17T - tank/archive/nas volmode default default tank/archive/nas filesystem_limit none default tank/archive/nas snapshot_limit none default tank/archive/nas filesystem_count none default tank/archive/nas snapshot_count none default tank/archive/nas redundant_metadata all default tank/archive/nas org.freebsd.ioc:active yes inherited from tank tank/archive/nas org.freenas:description archive inherited from tank/archive tank/archive/nas@auto-20191023.2031-100y type snapshot - tank/archive/nas@auto-20191023.2031-100y creation Wed Oct 23 20:31 2019 - tank/archive/nas@auto-20191023.2031-100y used 0 - tank/archive/nas@auto-20191023.2031-100y referenced 7.64T - tank/archive/nas@auto-20191023.2031-100y compressratio 1.02x - tank/archive/nas@auto-20191023.2031-100y devices on default tank/archive/nas@auto-20191023.2031-100y exec on default tank/archive/nas@auto-20191023.2031-100y setuid on default tank/archive/nas@auto-20191023.2031-100y xattr on default tank/archive/nas@auto-20191023.2031-100y version 5 - tank/archive/nas@auto-20191023.2031-100y utf8only off - tank/archive/nas@auto-20191023.2031-100y normalization none - tank/archive/nas@auto-20191023.2031-100y casesensitivity insensitive - tank/archive/nas@auto-20191023.2031-100y nbmand off default tank/archive/nas@auto-20191023.2031-100y primarycache all default tank/archive/nas@auto-20191023.2031-100y secondarycache all default tank/archive/nas@auto-20191023.2031-100y defer_destroy off - tank/archive/nas@auto-20191023.2031-100y userrefs 0 - tank/archive/nas@auto-20191023.2031-100y mlslabel - tank/archive/nas@auto-20191023.2031-100y refcompressratio 1.02x - tank/archive/nas@auto-20191023.2031-100y written 7.64T - tank/archive/nas@auto-20191023.2031-100y clones - tank/archive/nas@auto-20191023.2031-100y logicalreferenced 7.17T - tank/archive/nas@auto-20191023.2031-100y volmode default default tank/archive/nas@auto-20191023.2031-100y org.freebsd.ioc:active yes inherited from tank tank/archive/nas@auto-20191023.2031-100y org.freenas:description archive inherited from tank/archive
