SOLVED Riad group size lost 9TB

[Fei]

I use HGST 4TB * 32 create raidz2(6+2) *4 .
The raidz2(6+2)*4 with 4TB disk estimated capacity of 88.3 TB,but I use zfs list check size only 79.9 TB.
Where did 9TB go?

Code:

 pool: vol01
state: ONLINE
  scan: none requested
config:

    NAME                                            STATE     READ WRITE CKSUM
    vol01                                           ONLINE       0     0     0
      raidz2-0                                      ONLINE       0     0     0
        gptid/f22bdd95-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f278307c-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f2c3364a-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f30dd1a8-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f35971dc-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f3a38e92-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f3eedc98-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f43869e1-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
      raidz2-1                                      ONLINE       0     0     0
        gptid/f4859301-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f4d19457-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f51bf820-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f5701b0a-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f5c2d895-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f6127a39-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f661a30b-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f6ae648b-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
      raidz2-2                                      ONLINE       0     0     0
        gptid/f6fe7411-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f748e2c7-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f797cfa4-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f7e45698-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f83437e9-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f88214da-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f8d0095c-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f920de86-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
      raidz2-3                                      ONLINE       0     0     0
        gptid/f980338a-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/f9cf0d5c-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/fa1e3e0c-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/fa6d31f3-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/fabb8561-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/fb0c368e-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/fb5c9085-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0
        gptid/fbaca887-3bfe-11e5-894f-001b21261582  ONLINE       0     0     0

Code:

NAME   PROPERTY              VALUE                  SOURCE
vol01  type                  filesystem             -
vol01  creation              Thu Aug  6 13:50 2015  -
vol01  used                  11.1M                  -
vol01  available             79.9T                  -
vol01  referenced            213K                   -
vol01  compressratio         6.31x                  -
vol01  mounted               yes                    -
vol01  quota                 none                   local
vol01  reservation           none                   local
vol01  recordsize            128K                   default
vol01  mountpoint            /mnt/vol01             default

 

[danb35]

I'll bet the size there is reporting in TiB (2^40 bytes), not TB (10^12 bytes). That would account for the difference you're seeing.

 

[Fei]

I use "zpool list " print out 116TB( 32 Disk * 4 TB *0.91)

Code:

 <zpool list>
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
vol01          116T  16.1M   116T         -     0%     0%  1.00x  ONLINE  /mnt

then use "zfs list" print out 79.9TB <--- Why?

Code:

<zfs list >
NAME   USED  AVAIL  REFER  MOUNTPOINT
vol01    11.1M  79.9T    213K  /mnt/vol01

I think this not TB and TiB problem ,Because "zfs list " size = "zpool list" size - 8 parity *4TB *0.91 = 86.88 TB.
86.88-79.9=7.09TB this size too large ,so I think not TB and TiB problem.

 

[Fei]

If anyone want to know can refer this link
https://bugs.freenas.org/issues/10937