ZFS issue: Missing free space.

[rlaovai]

Greetings,

I have an issue with ZFS. Recently the ISCSI target stops to work, quick research revealed that parent dataset for zvol runs out of space (which is odd, because zvol is thick provisoned and 600-700G was left free on dataset pool1) the zpool it reporting to have free space left, but underlaying ZFS datssets reporting no free space available.

[root@freenas2]# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
pool1 40.9T 40.2T 654G 98% 1.00x ONLINE /mnt

[root@freenas2]# zfs list
NAME USED AVAIL REFER MOUNTPOINT
pool1 32.1T 0 230K /mnt/pool1
pool1/vol1 32.1T 0 32.1T -

As you can see zpool shows 654G of free space left, but zfs list showing zero space available.
No quota is set for pool1
Underlaying disks all green.

Is there any known way to reclaim the free space to pool1?

Best,
Alex.

 

[Bidule0hm]

Oh god, you've filled this pool to 98%. And you want to fill it more

We generally recommend to fill no more than 80% of the pool for classic usage and 50 to 60% max for iSCSI usage.

 

[Morbo]

Hi there,

I believe the internals of ZFS maintain a 1/64th of your pool size to ensure copy on write (CoW) functions are always able to succeed.

Looking at your figures the maths seems to line up pretty well:

40.9T * 1024 = 41881.6G
41881.6 / 64 = 654G

There's an old article by Ben Rockwood from Cuddletech (google it) that explains it.

Hope that helps,
Stuart

 

[Bidule0hm]

Yes, it's the 1/64 rule. I didn't thought it would be used one day because we are not supposed to fill a pool even close to 100 % but, yeah, it seems it's useful in the end :)

Cyberjock will love reading this thread :D