Snapshot Storage Usage Confusion (GUI vs CLI)

[ninjai]

So I can't find for the life of my why I'm almost out of storage. Now I just did find out. I was looking at my snap usage for my media dataset in the GUI and it seemed low. Especially considering I had deleted almost a TiB of data from it not long ago. This is what I see in the GUI:

I realize screenshot doesn't show all, but the "used" numbers look the same for the next page. Now when I use the CLI...:

That's almost a TiB! Why doesn't that show up in the GUI? And How can I find and delete the snapshots involed in eating that space?

 

[SweetAndLow]

There are multiple pages in the gui, did you try looking though all the pages?

 

[ninjai]

There are multiple pages in the gui, did you try looking though all the pages?

I realize screenshot doesn't show all, but the "used" numbers look the same for the next page.

Yes :(

Thing I don't get though is that if I use zfs list -t snap I get the same sort of output as the GUI. The one that says usedsnap is 962 G was the output of this command:

Code:

zfs list -o space -S usedsnap

 

[Apollo]

So I can't find for the life of my why I'm almost out of storage. Now I just did find out. I was looking at my snap usage for my media dataset in the GUI and it seemed low. Especially considering I had deleted almost a TiB of data from it not long ago. This is what I see in the GUI:
View attachment 30132
I realize screenshot doesn't show all, but the "used" numbers look the same for the next page. Now when I use the CLI...:
View attachment 30133

That's almost a TiB! Why doesn't that show up in the GUI? And How can I find and delete the snapshots involved in eating that space?

You have to be careful with how you interpret the table and the fields.
In a nutshell, you fisrt snapshot which I think not visible is holding 6.71TB of data hostage, which is good normally.
This is because "Storage/media@auto-20190405.2200-2w" content shows 0B used, meaning that it doesn't detect changes in Blocks and therefore doesn't need to hold the pointers to those blocks. "Used" can mean you have deleted files or added or renamed whatever. This will update the blocks and increase the "Used" space.
Field "Reference" is the actual amount of data held by the previous snapshots up to the listed snapshot.
If you want to recover your deleted 1TB, you will have to look which snapshot shows a reduction in storage locked by the snapshots.
In your case that would be:
"Storage/media@auto-20190313.2200-2w".
If you were to delete all the snapshot prior to "Storage/media@auto-20190313.2200-2w", you will be able to recover your 1TB and your pool capacity would drop around 5.76TB.

 

[fracai]

Decoding space used by snapshots is even harder than that. The USED column is only space unique to that snapshot. If you have three snapshots that all reference the same data as the dataset, they'll all say 0B. If you then delete 1TB from the dataset, the snapshots will still show 0B because they all share references to the same data. Delete the oldest snapshot and they remaining two will still say 0B. Delete the oldest again and the remaining snapshot will now jump to 1TB.

In your case, looking at the drop in REFERENCED data would probably identify what can be removed to free up a bunch of space. This gets harder when you haven't deleted a bunch of data, but changed a bunch. In that case, REFERENCED won't drop and you'll just have to prune old snapshots until you recover the space you need.

 

[ninjai]

Is it safe to blow away all of my snapshots to get a fresh slate without losing any data except for reference points between snapshots?

 

[fracai]

Destroying snapshots should not destroy your dataset. To be safe, maybe make a snapshot now and destroy all snapshots except that one. I wrote a collection of scripts to manage snapshots for me (different retention limits and removing 0B sized snapshots), but I'll occasionally go in and manually blow them away to free up space after deleting a large number of files. You just have to be careful you don't do this when you need to restore data from one of those snapshots (not that there's really a way to be sure of that...)