Quick questions about ZFS space usage

[gdarends]

Hi,

I received an alert that indicated that I was over 80% full on storage space on my ZFS pool.
So I went digging to find the issue. Turns out there was a .recycle folder that accumulated 892GB of storage.
Then I did a recursive remove on the folder, which is taking a while to remove.
But immediately in the FreeNAS interface I see the percentage drop to 68%.
How can it show that the space is available while it still had 700GB to delete still?

EDIT:
Oh and the dataset where I deleted the files, still shows 892GB used.

 

[Basil Hendroff]

It will be helpful if you provide some details of your FreeNAS hardware and software, and your pool configuration.

 

[gdarends]

It's in a RAIDZ1 configuration.
The hardware is a FreeNAS 1U Certified Server with 32GB RAM

 

[dlavigne]

Did this get resolved?

 

[gdarends]

No, it still shows that it's using almost 900GB of space.

CLI output:

Code:

[gd@nas /mnt/storage/ephemeral]$ du -h -d 1
 50G    ./ctp
4.2M    ./share
 50G    .

 

[dlavigne]

Don't use du as it isn't an accurate reflection of ZFS usage. zfs list should match the UI.

 

[gdarends]

Ok, here is the output. Maybe I just don't understand the workings of ZFS, but it shows USED 895GB and REFER is 50GB.
What is that 895GB?

Code:

[gd@nas /mnt/storage/ephemeral]$ zfs list -d 2
NAME                                                       USED  AVAIL  REFER  MOUNTPOINT
storage/ephemeral                                          895G  1.60T   140K  /mnt/storage/ephemeral
storage/ephemeral/ctp                                      895G  1.60T  49.8G  /mnt/storage/ephemeral/ctp
storage/ephemeral/share                                   4.38M   100G  4.38M  /mnt/storage/ephemeral/share

 

[Mlovelace]

Ok, here is the output. Maybe I just don't understand the workings of ZFS, but it shows USED 895GB and REFER is 50GB.
What is that 895GB?

Code:

[gd@nas /mnt/storage/ephemeral]$ zfs list -d 2
NAME                                                       USED  AVAIL  REFER  MOUNTPOINT
storage/ephemeral                                          895G  1.60T   140K  /mnt/storage/ephemeral
storage/ephemeral/ctp                                      895G  1.60T  49.8G  /mnt/storage/ephemeral/ctp
storage/ephemeral/share                                   4.38M   100G  4.38M  /mnt/storage/ephemeral/share

Do you have snapshots on the dataset in question?

Code:

zfs list -t snapshot

will display them if you do.

 

[Heracles]

Hi G Arends,

Did you reserved space for your dataset ? Here, because my main NAS and DR NAS are not the same size, I created a dataset that contains everything to be replicated to the DR site. I configured it with both MAX SIZE and RESERVED SIZE to be sure it will never go above the capacity of the receiveing DR server. I am also sure that none of the second usage I do in the pool may reduce the capacity available in the HA dataset.

Because I reserved some space for that dataset, that space is marked as USED by ZFS despite there is no actual data in it. I reserved about 7 TB but I am using only about 1 as of now. Still, the dataset is said to use all of the 7 TB because these TB are reserved, so not available for anything else.

 

[gdarends]

Snapshots don't seem to be an issue.

Code:

[gd@nas ~]$ zfs list -t snapshot
NAME                                                                     USED  AVAIL  REFER  MOUNTPOINT
storage/ephemeral/ctp@manual-20160405                                   2.95G      -  2.95G  -
storage/ephemeral/ctp@auto-20190318.0900-1w                             15.5M      -  48.0G  -
storage/ephemeral/ctp@auto-20190319.0900-1w                             15.5M      -  48.0G  -
storage/ephemeral/ctp@auto-20190320.0900-1w                             36.9M      -  48.5G  -
storage/ephemeral/ctp@auto-20190321.0900-1w                             41.1M      -  49.8G  -
storage/ephemeral/ctp@auto-20190322.0900-1w                              166M      -  51.3G  -

 

[gdarends]

Ok.... so I just checked the FreeNAS interface and it seems it fixed itself. Usage also dropped again to 53%.
I haven't made any changes. I still don't understand what happened.

Edit: Also, what's the deal with 0.53%? It should be 53%.