Missing Disk Space

[jbear]

Hi,

I'm trying to find out, why there's a discrepancy between the free space in my zpool and the usable space of the mounted dataset. According to "zpool list" there is 2.62TB free space in the pool:

[root@freenas] ~# zpool list

NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT

tank1 9.06T 6.44T 2.62T 71% 1.00x ONLINE /mnt


But "zfs list" and "df -h" only report 1,5 TB of free space availabe:

[root@freenas] ~# zfs list

NAME USED AVAIL REFER MOUNTPOINT

tank1 3.81T 1.47T 14.3M /mnt/tank1

tank1/.system 118M 1.47T 312K /mnt/tank1/.system

tank1/.system/cores 12.3M 1.47T 12.3M /mnt/tank1/.system/cores

tank1/.system/rrd 256K 1.47T 256K /mnt/tank1/.system/rrd

tank1/.system/samba4 7.68M 1.47T 7.68M /mnt/tank1/.system/samba4

tank1/.system/syslog 97.0M 1.47T 97.0M /mnt/tank1/.system/syslog

tank1/jails 284K 1.47T 284K /mnt/tank1/jails

tank1/linux 967G 1.47T 939G /mnt/tank1/linux

tank1/shadowprotect 2.87T 1.47T 2.87T /mnt/tank1/shadowprotect


[root@freenas] ~# df -h

Filesystem Size Used Avail Capacity Mounted on

/dev/ufs/FreeNASs1a 926M 703M 148M 83% /

devfs 1.0k 1.0k 0B 100% /dev

/dev/md0 4.6M 3.3M 901k 79% /etc

/dev/md1 823k 2.0k 756k 0% /mnt

/dev/md2 149M 20M 116M 15% /var

/dev/ufs/FreeNASs4 19M 5.5M 12M 30% /data

tank1 1.5T 14M 1.5T 0% /mnt/tank1

tank1/.system 1.5T 312k 1.5T 0% /mnt/tank1/.system

tank1/.system/cores 1.5T 12M 1.5T 0% /mnt/tank1/.system/cores

tank1/.system/rrd 1.5T 255k 1.5T 0% /mnt/tank1/.system/rrd

tank1/.system/samba4 1.5T 7.7M 1.5T 0% /mnt/tank1/.system/samba4

tank1/.system/syslog 1.5T 97M 1.5T 0% /mnt/tank1/.system/syslog

tank1/jails 1.5T 284k 1.5T 0% /mnt/tank1/jails

tank1/linux 2.4T 939G 1.5T 38% /mnt/tank1/linux

tank1/shadowprotect 4.3T 2.9T 1.5T 66% /mnt/tank1/shadowprotect



Shouldn't there be 2.62TB available in the dataset?

There's no quota on the dataset and the only snapshot uses 27.7GB:

[root@freenas] ~# zfs get quota tank1

NAME PROPERTY VALUE SOURCE

tank1 quota none default

[root@freenas] ~# zfs list -t snapshot

NAME USED AVAIL REFER MOUNTPOINT

tank1/linux@backup 27.7G - 856G -


Where are the missing 1.1 TB?

Thanks in advance!

 

[jgreco]

The pool number refers to RAW disk space that isn't used. The df numbers refer to the space actually used/available for use.

 

[jbear]

The pool number refers to RAW disk space that isn't used. The df numbers refer to the space actually used/available for use.

Agreed, but that doesn't explain this huge discrepancy. 2.6 TB of free blocks in the pool should translate to a lot more free bytes on the mounted dataset, shouldn't it? 1.1 TB of 2.6 TB are missing. That's 42%. That can't be metadata overhead, can it?

 

[jgreco]

Let me explain it a different way.

[root@freenas] ~# zpool list

NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT

tank1 9.06T 6.44T 2.62T 71% 1.00x ONLINE /mnt

You are using 71% your raw disk space.

[root@freenas] ~# zfs list

NAME USED AVAIL REFER MOUNTPOINT

tank1 3.81T 1.47T 14.3M /mnt/tank1

This is usable space; used + available sums up to 5.28TB. If you then look at 3.81TB of 5.28TB, that shows that you have 72% of your usable space used.

These numbers are consistent.

When you create a RAIDZ or mirror or whatever, you lose space to be able to protect your data. Now I'm pretty lazy and I am not going to go through the combinations of possible disk layouts to determine what you might have, and since you didn't provide that info, all that I am really going to do is look at the raw versus usable numbers and notice that the percentage is consistent.

You do have 2.62TB of space free, but when you allocate blocks for parity, you will only find yourself able to store 1.47TB of actual user data within that space.[/QUOTE]

 

[jbear]

Ah, I got it now! I simply wasn't aware that "zpool list" reports the total size including the parity blocks. I assumed it shows the usable size of the pool.

 

[jgreco]

Ah, I got it now! I simply wasn't aware that "zpool list" reports the total size including the parity blocks. I assumed it shows the usable size of the pool.

It's actually even a little more complicated than that, but for normal use it works out more or less the way you'd expect.

And I bet you just felt that very disappointing feeling people get when they realize they don't have quite as much space as they were hoping they had, right?

Anyways, there ya go, sorry to break it to you.