lz4 compression ratios, and zvols - how do I utilize space savings from compression?

ee21

Dabbler
Joined
Nov 2, 2020
Messages
33
Okay, so from reading another thread, my current understanding of the compression ratio shown in storage view, is basically summed up by this quote from danb35:
"A number of 1.09x means that you're storing 1.09x as much data as you're using drive space--you're using 1000 MB of pool space to store 1090 MB of data" - taken from this thread here: https://www.truenas.com/community/t...pression-ratio-numbers-in-freenas-mean.42330/

If this were a dataset, and NFS share we were talking about, the above would be pretty straight forward in my head.. I am using zvols and iSCSI shares unfortunately, and as such I am a little lost here. Follow me for a moment, and see if someone can explain this a little bit to me:

  • I have a pool with 1500GB available
  • I create a zvol sized at 100GB (not thin provisioned)
  • My pool now shows 1400GB available, 100GB used
  • The zvol shows 100GB used 1500GB available (counts the available space on the pool too apparently)
  • I create an iSCSI share, with a device-type extent mapped to the zvol I just created, and attached it to a Windows initiator
  • From the Windows initiator, I copy a 20GB VHDX file to the drive, after formatting it in disk manager
  • Windows shows I have 20GB used, and 80GB free on the partition
  • TrueNAS shows the zvol has 100GB used still, but 1488GB available, and compression ratio 1.55.
It would appear that the 20GB VHDX is occupying only about 12GB of space, which is great!... But Windows sees that 20GB is taken, and only 80GB left on the zvol - even if it really is 88GB free technically?..

So hypothetically now, I grow my VM a ton, and now I'm sitting at 100GB of 100GB on my Windows partition, still with a 1.5x compression ratio, which leaves me a zvol that shows me 100GB used, and 1434GB free (approximately). Great. Subtract the 1400GB left from the pool, and the zvol should still have about 34GB available.. Windows doesn't see it that way, and am forced to extend C partition, but cannot as no more space is being presented.

How do I fix this so I can take advantage of the compression's space savings? Use thin provisioned ("sparse") zvols?
I'm not super big on that, but if it works it works.. My problem with that is, then those zvols would need to be overprovisioned technically. So if I wanted to actually use (80% of) my entire 1500GB pool's actual disk space, assuming a 1.5x compression ratio, I would need to provision a zvol that was (80% of) 2250 GB - if TrueNAS will even allow this! - EDIT: Seems it will using the force size flag, but still not sure its a good idea! Any advice appreciated!
 
Last edited:

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,112
Short answer: Yes, you need to use sparse zvols and oversubscribe your pool storage. Monitor carefully, here thar be dragons.

Long answer: Do it in smaller chunks. Instead of a single 1500GB thick zvol consuming 100% of the 1500GB pool, start with a single 500GB sparse zvol. Add files/VMDKs/VHDXs until it gets close to full - but don't oversubscribe at that level. At this point, you will also know how well your data compresses. Let's say it give you this nice 1.5x number - if you cut off at 450GB (logical) on the zvol, it uses 300GB (physical) and you have 1200GB left.

You can repeat this three more times (4x500G logical) and if your data stays compressed at 1.5x, you get to store that 450x4 = 1800G into 1200G of space. Presto, you're at 80% used on your SSDs.
 

ee21

Dabbler
Joined
Nov 2, 2020
Messages
33
Thanks! I've got a decent feel for how things compress with the way I want now. And no, I wasn't going with a single 1500GB zvol, that was just an example haha.. Right now I have a bunch of 80GB thick provisioned volumes, I likely will keep the same structure with thinly provisioned ones.
 
Top