Dedup data - Decrease How?

[ThisTruenasUser]

Hi all.
Firstly for context, I am aware of how some dislike deduplication.
For me it is the only reason I have to use truenas.
The purpose being multiple ISCSI devices for my big steam games collection.


I understand it can be very resource consuming if mis-configured.

It is being tested on an old machine i7-3770, 43GB ram, 1TB hd with 16GB optane as dedup drive.

Due to the mysterious 128k maximum block size in truenas , had to create the zvols with 1MB block sizes using the command line. Dedup on, no compression & sparse.
Two were created, & both shared over ISCSI.
On the windows machine, connected to the ISCSI shares and formatted ntfs, with 1MB block sizes.

So copied multiple copies of various games to each drive, and the dedup value was 2.00, which is great and the actual dedup data was very low.
It fitted with the X GB dedup data for the X TB storage well.

I deleted all the data in the volumes over ISCSI.
In truenas, the data usage of the pool decreased as expected.

The dedup data did not.
It has stayed the same.
My guess is the blocks are just marked a free, and the data is not deleted.
Is that correct?
I assume the data is still there in the zvol?
It is odd though as the pool has almost nothing in the way of usage.

If not then as time goes on, dedup data usage with head off to infinity!!!

I think that is what happen when deleted files in windows.

I intend to setup a truenas scale machine with:
3600X CPU, 64GB RAM, 2.5Gb network card.
5 x 4TB (CMR) raidz1 with 2 x 16GB intel optane for deduplication.

A few of virtual machines , low spec windows, linux & ubuntu server for streaming. I will not be using the truenas scale apps.
A couple of ISCSI shares (dedup on) for games storage & windows shares (dedup off) for media. those two shares will have the exact same games on there.

So I need to get the deduplication to work correctly, otherwise will need to buy more storage for games. Perhaps an 8TB cheapish smr drive for games.
In that situation have little use for truenas.

To sum up, is it when unique data is replaced for a block in a zvol, the dedup data is also replaced?

Useful info appreciated!

Thanks

 

[ThisTruenasUser]

Hi I meant:

To sum up, is it when unique data is replaced for a block in a all dedup z-voumes, the dedup data is also replaced?

 

[morganL]

It's probably a slow process because its a CoW file system. When you "delete" data, you don't actually delete it immediately. The disk space just becomes free, but the data is not changed. The metadata is changed, but is not immediately removed. The only way to really test is tio write new data and see that you have enough spare capacity in the pool.

Trust the usage data on the pool.