Free space discrepancy

[IanWorthington]

Anyone know why there's a discrepancy between the free space returned by zpool list and that by zfs list?

(Configuration is 8x4TB + 3x4TB raidz3)

Code:

% zpool list
NAME      SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
VOLUME1    40T  31.5T  8.47T    78%  1.00x  ONLINE  /mnt

% zfs list
NAME          USED  AVAIL  REFER  MOUNTPOINT
VOLUME1      22.9T  5.70T   581K  /mnt/VOLUME1
.
.
.

 

[cyberjock]

One considers your redundancy and one doesn't.

 

[IanWorthington]

Thanks CJ, but not seeing how that works.

Surely 11x4TB in z3 should give me a size about 44TB with 32TB available? The ALLOC 31.5T figure seems to be close enough to 32TB for me to assume that that's right. But USED+AVAIL=28.6, far short of 32TB.

What am I missing here?

i

 

[cyberjock]

Your assessment is wrong because 4TB drives are NOT actually 4TB.

Things are much more complex than they look. Sorry but I'm not in a position to rehash this out right now. I explained this is detail several times before. Please search for other threads on this topic if you want to understand what is going on behind the scenes.

 

[Whattteva]

I may be wrong on this, but I'll attempt to clarify for CJ (correct me if I'm wrong).
I think what he is talking about is the fact that OS entities (Microsoft, Linux, Apple, BSD, etc...) have a disagreement about the definition of each of the size prefix k, M, G, T and so forth with the memory and hard drive manufacturers.
OS's see these prefixes as powers of 2 like so:

1. kilo = 2^10 = 1024^1 = 1024
   
2. Mega = 2^20 = 1024^2 = 1'048'576
3. Giga = 2^30 = 1024^3 = 1'073'741'824
4. Terra = 2^40 = 1024^4 = 1'099'511'627'776

Memory, hard drive, and communications (NIC, modems, etc.) manufacturers, on the other hand see those prefixes as exact powers of 10 like so:

1. kilo = 10^3 = 1000^1 = 1000
   
2. Mega = 10^6 = 1000^2 = 1'000'000
3. Giga = 10^9 = 1000^3 = 1'000'000'000
4. Terra = 10^12 = 1000^4 = 1'000'000'000'000

Hence, the discrepancies. Basically, the higher up you go in capacity, the more it will appear that you are "losing" capacity due to that rounding.
In your case, your "expected 32 TB" is really only seen by the OS as: 32'000'000'000'000 / 1024^4 ~ 29.1038 TB.
Furthermore, I believe you also lose some space incurred by the overhead of using the filesystem due to block size (not sure the exact formula for this, someone else can probably pitch in).
That being said, doesn't your number now make a lot more sense? BTW, this isn't specific to FreeNAS. You will observe this regardless of the filesystem or OS you use.