Can't see a > 2 TB folder

Kenshin9977

Cadet
Joined
Feb 13, 2020
Messages
6
I have 8 TB of storage and recently saw I had only < 2 TB of free space so I decided it was time to delete a few things.
While searching for the biggest files and folders to delete I understood there is something weird.
I have a dataset that occupies 4.8 TB, within it is my main datasat which occupies 2.2 TB. The other files and folders within the 4 TB dataset only take around 200 GB.
So what is taking more than 2 TB and that I can't see ?
I tried running zpool list, zfs list, testparm and ls -la /mnt/storage/NASLoup to better understand the issue but to no avail.
Here is the output of those commands
Code:
root@freenas[~]# zpool list
NAME           SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
freenas-boot  29.5G  20.4G  9.11G        -         -    35%    69%  1.00x    ONLINE  -
storage       14.5T  10.1T  4.38T        -         -    16%    69%  1.00x    ONLINE  /mnt
root@freenas[~]# zpool status
  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0B in 00:00:17 with 0 errors on Fri May 14 03:45:17 2021
config:

        NAME          STATE     READ WRITE CKSUM
        freenas-boot  ONLINE       0     0     0
          ada0p2      ONLINE       0     0     0

errors: No known data errors

  pool: storage
 state: ONLINE
  scan: scrub repaired 0B in 08:23:42 with 0 errors on Sun May 16 08:23:44 2021
config:

        NAME                                            STATE     READ WRITE CKSUM
        storage                                         ONLINE       0     0 0
          raidz2-0                                      ONLINE       0     0 0
            gptid/687615ef-6cd6-11eb-ad0e-e0d55e83f759  ONLINE       0     0 0
            gptid/1d7bbe35-6d31-11eb-8672-e0d55e83f759  ONLINE       0     0 0
            gptid/0d811d23-6d94-11eb-a679-e0d55e83f759  ONLINE       0     0 0
            gptid/8f695cb5-6df0-11eb-b9bf-e0d55e83f759  ONLINE       0     0 0

errors: No known data errors
root@freenas[~]# zfs list
NAME                                                       USED  AVAIL     REFER  MOUNTPOINT
freenas-boot                                              20.4G  8.19G       64K  none
freenas-boot/ROOT                                         20.4G  8.19G       29K  none
freenas-boot/ROOT/11.2-U7                                  458K  8.19G      759M  /
freenas-boot/ROOT/11.3-RELEASE                             250K  8.19G     1.04G  /
freenas-boot/ROOT/11.3-U5                                  244K  8.19G     1.05G  /
freenas-boot/ROOT/FreeNAS-12.0-U3.1                       20.4G  8.19G     16.9G  /
freenas-boot/ROOT/Initial-Install                            1K  8.19G      755M  legacy
freenas-boot/ROOT/default                                  176K  8.19G      755M  legacy
storage                                                   4.99T  1.82T      140K  /mnt/storage
storage/.bhyve_containers                                 70.5M  1.82T     70.5M  /mnt/storage/.bhyve_containers
storage/.system                                            190M  1.82T      151K  legacy
storage/.system/configs-c2484c3f88124e51b79845e4fb993a70   122M  1.82T      122M  legacy
storage/.system/cores                                      128K  1024M      128K  legacy
storage/.system/rrd-c2484c3f88124e51b79845e4fb993a70      58.6M  1.82T     58.6M  legacy
storage/.system/samba4                                     965K  1.82T      541K  legacy
storage/.system/services                                   140K  1.82T      140K  legacy
storage/.system/syslog-c2484c3f88124e51b79845e4fb993a70   7.29M  1.82T     7.29M  legacy
storage/.system/webui                                      128K  1.82T      128K  legacy
storage/Alpinev2-d6x5gq                                   10.2G  1.83T     81.4K  -
storage/Debian-0l4yn4                                     9.69G  1.83T     81.4K  -
storage/NASLoup                                           4.85T  1.82T     2.67T  /mnt/storage/NASLoup
storage/NASLoup/NAS_Torrent                               2.19T  1.82T     2.18T  /mnt/storage/NASLoup/NAS_Torrent
storage/NASLoup/NAS_Torrent/Musiques                      8.08G  1.82T     8.08G  /mnt/storage/NASLoup/NAS_Torrent/Musiques
storage/Ombi-qio5l6                                       9.69G  1.83T     81.4K  -
storage/Ombi-zr2xil                                       9.69G  1.83T      272M  -
storage/iocage                                            96.7G  1.82T     16.4M  /mnt/storage/iocage
storage/iocage/Alpine-8yfkv                               10.2G  1.82T     7.07G  -
storage/iocage/Alpine-ujs7tg                              10.2G  1.83T     81.4K  -
storage/iocage/AlpineExtended-zmfhc                       10.2G  1.83T     81.4K  -
storage/iocage/AlpineVirt-bmh1r                           10.2G  1.83T     7.86M  -
storage/iocage/Alpinev3-qnoub                             10.2G  1.83T      665M  -
storage/iocage/download                                   11.7G  1.82T      328M  /mnt/storage/iocage/download
storage/iocage/download/11.1-RELEASE                       119M  1.82T      119M  /mnt/storage/iocage/download/11.1-RELEASE
storage/iocage/download/11.2-RELEASE                       125M  1.82T      125M  /mnt/storage/iocage/download/11.2-RELEASE
storage/iocage/download/11.3-RELEASE                       289M  1.82T      289M  /mnt/storage/iocage/download/11.3-RELEASE
storage/iocage/download/11.4-RELEASE                       306M  1.82T      306M  /mnt/storage/iocage/download/11.4-RELEASE
storage/iocage/download/12.0-RELEASE                       206M  1.82T      206M  /mnt/storage/iocage/download/12.0-RELEASE
storage/iocage/download/12.2-RELEASE                       238M  1.82T      238M  /mnt/storage/iocage/download/12.2-RELEASE
storage/iocage/download/Debian10-bf7spj                   10.2G  1.82T     4.07G  -
storage/iocage/images                                      128K  1.82T      128K  /mnt/storage/iocage/images
storage/iocage/jails                                      28.4G  1.82T      174K  /mnt/storage/iocage/jails
storage/iocage/jails/BazarrV2                              932M  1.82T      134K  /mnt/storage/iocage/jails/BazarrV2
storage/iocage/jails/BazarrV2/root                         932M  1.82T     1.47G  /mnt/storage/iocage/jails/BazarrV2/root
storage/iocage/jails/Jackett                              3.30G  1.82T      151K  /mnt/storage/iocage/jails/Jackett
storage/iocage/jails/Jackett/root                         3.30G  1.82T     2.08G  /mnt/storage/iocage/jails/Jackett/root
storage/iocage/jails/Lidarrv2                             6.40G  1.82T      151K  /mnt/storage/iocage/jails/Lidarrv2
storage/iocage/jails/Lidarrv2/root                        6.40G  1.82T     6.47G  /mnt/storage/iocage/jails/Lidarrv2/root
storage/iocage/jails/OrganizrV2                            471M  1.82T      157K  /mnt/storage/iocage/jails/OrganizrV2
storage/iocage/jails/OrganizrV2/root                       471M  1.82T     1.23G  /mnt/storage/iocage/jails/OrganizrV2/root
storage/iocage/jails/Radarr_Sonarr_Transmission_VPN       5.82G  1.82T      145K  /mnt/storage/iocage/jails/Radarr_Sonarr_Transmission_VPN
storage/iocage/jails/Radarr_Sonarr_Transmission_VPN/root  5.82G  1.82T     5.90G  /mnt/storage/iocage/jails/Radarr_Sonarr_Transmission_VPN/root
storage/iocage/jails/Stress CPU                           79.0M  1.82T      163K  /mnt/storage/iocage/jails/Stress CPU
storage/iocage/jails/Stress CPU/root                      78.9M  1.82T      482M  /mnt/storage/iocage/jails/Stress CPU/root
storage/iocage/jails/TautulliV2                            439M  1.82T      140K  /mnt/storage/iocage/jails/TautulliV2
storage/iocage/jails/TautulliV2/root                       438M  1.82T     1010M  /mnt/storage/iocage/jails/TautulliV2/root
storage/iocage/jails/Unpackerr                            86.7M  1.82T      151K  /mnt/storage/iocage/jails/Unpackerr
storage/iocage/jails/Unpackerr/root                       86.6M  1.82T      883M  /mnt/storage/iocage/jails/Unpackerr/root
storage/iocage/jails/freeplex                             6.25G  1.82T      157K  /mnt/storage/iocage/jails/freeplex
storage/iocage/jails/freeplex/root                        6.25G  1.82T     4.62G  /mnt/storage/iocage/jails/freeplex/root
storage/iocage/jails/mineos                               4.28G  1.82T      349K  /mnt/storage/iocage/jails/mineos
storage/iocage/jails/mineos/root                          4.28G  1.82T     4.34G  /mnt/storage/iocage/jails/mineos/root
storage/iocage/jails/reverse-proxy                         400M  1.82T      151K  /mnt/storage/iocage/jails/reverse-proxy
storage/iocage/jails/reverse-proxy/root                    399M  1.82T      798M  /mnt/storage/iocage/jails/reverse-proxy/root
storage/iocage/log                                         221K  1.82T      221K  /mnt/storage/iocage/log
storage/iocage/releases                                   5.79G  1.82T      128K  /mnt/storage/iocage/releases
storage/iocage/releases/11.1-RELEASE                       376M  1.82T      128K  /mnt/storage/iocage/releases/11.1-RELEASE
storage/iocage/releases/11.1-RELEASE/root                  376M  1.82T      376M  /mnt/storage/iocage/releases/11.1-RELEASE/root
storage/iocage/releases/11.2-RELEASE                       416M  1.82T      128K  /mnt/storage/iocage/releases/11.2-RELEASE
storage/iocage/releases/11.2-RELEASE/root                  415M  1.82T      404M  /mnt/storage/iocage/releases/11.2-RELEASE/root
storage/iocage/releases/11.3-RELEASE                      1.68G  1.82T      128K  /mnt/storage/iocage/releases/11.3-RELEASE
storage/iocage/releases/11.3-RELEASE/root                 1.68G  1.82T     1.31G  /mnt/storage/iocage/releases/11.3-RELEASE/root
storage/iocage/releases/11.4-RELEASE                      1.83G  1.82T      128K  /mnt/storage/iocage/releases/11.4-RELEASE
storage/iocage/releases/11.4-RELEASE/root                 1.83G  1.82T     1.43G  /mnt/storage/iocage/releases/11.4-RELEASE/root
storage/iocage/releases/12.0-RELEASE                       750M  1.82T      140K  /mnt/storage/iocage/releases/12.0-RELEASE
storage/iocage/releases/12.0-RELEASE/root                  750M  1.82T      750M  /mnt/storage/iocage/releases/12.0-RELEASE/root
storage/iocage/releases/12.2-RELEASE                       799M  1.82T      140K  /mnt/storage/iocage/releases/12.2-RELEASE
storage/iocage/releases/12.2-RELEASE/root                  799M  1.82T      799M  /mnt/storage/iocage/releases/12.2-RELEASE/root
storage/iocage/templates                                   128K  1.82T      128K  /mnt/storage/iocage/templates
storage/webDB                                              128K  1.82T      128K  /mnt/storage/webDB
root@freenas[~]# testparm
Load smb config files from /usr/local/etc/smb4.conf
Loaded services file OK.
Server role: ROLE_STANDALONE

Press enter to see a dump of your service definitions

# Global parameters
[global]
        aio max threads = 2
        bind interfaces only = Yes
        disable spoolss = Yes
        dns proxy = No
        enable web service discovery = Yes
        kernel change notify = No
        load printers = No
        logging = file
        max log size = 5120
        nsupdate command = /usr/local/bin/samba-nsupdate -g
        registry shares = Yes
        restrict anonymous = 2
        server role = standalone server
        server string = FreeNAS Server
        unix extensions = No
        idmap config *: range = 90000001-100000000
        idmap config * : backend = tdb
        directory name cache size = 0
        dos filemode = Yes


[NASLoup]
        ea support = No
        kernel share modes = No
        path = /mnt/storage/NASLoup
        posix locking = No
        read only = No
        vfs objects = streams_xattr shadow_copy_zfs ixnas aio_fbsd
        nfs4:chown = true

root@freenas[~]# ls -la /mnt/storage/NASLoup
total 1489004
drwx-wx--x+  5 new_media  new_media            8 May 18 19:37 .
drwxr-xr-x   6 root       wheel                6 May 18 20:27 ..
drwx-----T+ 12 new_media  new_media           14 May 18 19:38 Audrey Backup
drwxr-x--x+ 16 new_media  new_media           16 May 18 20:07 NAS_Torrent
-rwx-----T+  1 new_media  new_media  21474836480 Feb 11  2020 ombiraw_Ombi
-rwx-----T+  1 new_media  new_media  21474836480 Feb 11  2020 OmbiRaw_OmbiDocker
drwx-----T+  4 new_media  new_media            4 May 18 19:42 OnePlus One Backup
-rwx-----T+  1 new_media  new_media           95 Feb 12  2020 traefik.toml

Here is the content of NASLoup as seen by WizTree through SMB ( du took way longer than WizTree to get the size of NASLoup)
WizTree64_e742LBOlET.png
 
Joined
Oct 22, 2019
Messages
3,641
Maybe all snapshots combined are taking up nearly 2TB?

You can check with this:
zfs get -r -t filesystem usedsnap storage

Look through this list. It will print out each line as the dataset (all that live under the pool "storage") and only show you the space being taken up by the snapshots (not the filesystem/dataset itself). For datasets that have no snapshots or only empty ones, this will show as "0B". For those with cumulative snapshots, it's possible this number can be quite large, in terms of several or hundreds of GB, especially if over time you've deleted many files and made big changes ever since taking a very early snapshot while a dataset was still young, yet filled with tons of data.
 

Kenshin9977

Cadet
Joined
Feb 13, 2020
Messages
6
Thank you for the help. Sadly it doesn't amount to a lot

Code:
root@freenas[~]# ls -la /mnt/storage/NASLoup
total 1489004
drwx-wx--x+  5 new_media  new_media            8 May 18 19:37 .
drwxr-xr-x   6 root       wheel                6 May 18 20:27 ..
drwx-----T+ 12 new_media  new_media           14 May 18 19:38 Audrey Backup
drwxr-x--x+ 16 new_media  new_media           16 May 18 20:07 NAS_Torrent
-rwx-----T+  1 new_media  new_media  21474836480 Feb 11  2020 ombiraw_Ombi
-rwx-----T+  1 new_media  new_media  21474836480 Feb 11  2020 OmbiRaw_OmbiDocker
drwx-----T+  4 new_media  new_media            4 May 18 19:42 OnePlus One Backup
-rwx-----T+  1 new_media  new_media           95 Feb 12  2020 traefik.toml
root@freenas[~]# zfs get -r -t filesystem usedsnap storage
NAME                                                      PROPERTY         VALUE     SOURCE
storage                                                   usedbysnapshots  0B     -
storage/.bhyve_containers                                 usedbysnapshots  0B     -
storage/.system                                           usedbysnapshots  0B     -
storage/.system/configs-c2484c3f88124e51b79845e4fb993a70  usedbysnapshots  0B     -
storage/.system/cores                                     usedbysnapshots  0B     -
storage/.system/rrd-c2484c3f88124e51b79845e4fb993a70      usedbysnapshots  0B     -
storage/.system/samba4                                    usedbysnapshots  424K     -
storage/.system/services                                  usedbysnapshots  0B     -
storage/.system/syslog-c2484c3f88124e51b79845e4fb993a70   usedbysnapshots  0B     -
storage/.system/webui                                     usedbysnapshots  0B     -
storage/NASLoup                                           usedbysnapshots  0B     -
storage/NASLoup/NAS_Torrent                               usedbysnapshots  0B     -
storage/NASLoup/NAS_Torrent/Musiques                      usedbysnapshots  0B     -
storage/iocage                                            usedbysnapshots  0B     -
storage/iocage/download                                   usedbysnapshots  0B     -
storage/iocage/download/11.1-RELEASE                      usedbysnapshots  0B     -
storage/iocage/download/11.2-RELEASE                      usedbysnapshots  0B     -
storage/iocage/download/11.3-RELEASE                      usedbysnapshots  0B     -
storage/iocage/download/11.4-RELEASE                      usedbysnapshots  0B     -
storage/iocage/download/12.0-RELEASE                      usedbysnapshots  0B     -
storage/iocage/download/12.2-RELEASE                      usedbysnapshots  0B     -
storage/iocage/images                                     usedbysnapshots  0B     -
storage/iocage/jails                                      usedbysnapshots  0B     -
storage/iocage/jails/BazarrV2                             usedbysnapshots  0B     -
storage/iocage/jails/BazarrV2/root                        usedbysnapshots  0B     -
storage/iocage/jails/Jackett                              usedbysnapshots  215K     -
storage/iocage/jails/Jackett/root                         usedbysnapshots  1.30G     -
storage/iocage/jails/Lidarrv2                             usedbysnapshots  0B     -
storage/iocage/jails/Lidarrv2/root                        usedbysnapshots  0B     -
storage/iocage/jails/OrganizrV2                           usedbysnapshots  0B     -
storage/iocage/jails/OrganizrV2/root                      usedbysnapshots  0B     -
storage/iocage/jails/Radarr_Sonarr_Transmission_VPN       usedbysnapshots  0B     -
storage/iocage/jails/Radarr_Sonarr_Transmission_VPN/root  usedbysnapshots  0B     -
storage/iocage/jails/Stress CPU                           usedbysnapshots  0B     -
storage/iocage/jails/Stress CPU/root                      usedbysnapshots  0B     -
storage/iocage/jails/TautulliV2                           usedbysnapshots  0B     -
storage/iocage/jails/TautulliV2/root                      usedbysnapshots  0B     -
storage/iocage/jails/Unpackerr                            usedbysnapshots  0B     -
storage/iocage/jails/Unpackerr/root                       usedbysnapshots  0B     -
storage/iocage/jails/freeplex                             usedbysnapshots  221K     -
storage/iocage/jails/freeplex/root                        usedbysnapshots  1.99G     -
storage/iocage/jails/mineos                               usedbysnapshots  0B     -
storage/iocage/jails/mineos/root                          usedbysnapshots  0B     -
storage/iocage/jails/reverse-proxy                        usedbysnapshots  0B     -
storage/iocage/jails/reverse-proxy/root                   usedbysnapshots  0B     -
storage/iocage/log                                        usedbysnapshots  0B     -
storage/iocage/releases                                   usedbysnapshots  0B     -
storage/iocage/releases/11.1-RELEASE                      usedbysnapshots  0B     -
storage/iocage/releases/11.1-RELEASE/root                 usedbysnapshots  0B     -
storage/iocage/releases/11.2-RELEASE                      usedbysnapshots  0B     -
storage/iocage/releases/11.2-RELEASE/root                 usedbysnapshots  11.0M     -
storage/iocage/releases/11.3-RELEASE                      usedbysnapshots  0B     -
storage/iocage/releases/11.3-RELEASE/root                 usedbysnapshots  377M     -
storage/iocage/releases/11.4-RELEASE                      usedbysnapshots  0B     -
storage/iocage/releases/11.4-RELEASE/root                 usedbysnapshots  413M     -
storage/iocage/releases/12.0-RELEASE                      usedbysnapshots  0B     -
storage/iocage/releases/12.0-RELEASE/root                 usedbysnapshots  0B     -
storage/iocage/releases/12.2-RELEASE                      usedbysnapshots  0B     -
storage/iocage/releases/12.2-RELEASE/root                 usedbysnapshots  279K     -
storage/iocage/templates                                  usedbysnapshots  0B     -
storage/webDB                                             usedbysnapshots  0B     -
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
Try cd /mnt/storage/NASLoup && du -sh * .??* please.
 

Kenshin9977

Cadet
Joined
Feb 13, 2020
Messages
6
The command doesn't seem to work, so instead I used du -sh *.
Code:
root@freenas[~]# cd /mnt/storage/NASLoup && du -sh * .??*
zsh: no matches found: .??*
root@freenas[/mnt/storage/NASLoup]# du -sh *
 96G    Audrey Backup
1.6T    NAS_Torrent
418M    ombiraw_Ombi
1.0G    OmbiRaw_OmbiDocker
 16G    OnePlus One Backup
512B    traefik.toml

zfs tells me NASLoup takes over 4 TB, which doesn't match what du sees
Code:
root@freenas[~]# zfs list -o name,used -d1
NAME                        USED
freenas-boot               20.4G
freenas-boot/ROOT          20.4G
storage                    4.31T
storage/.bhyve_containers  70.5M
storage/.system             190M
storage/Alpinev2-d6x5gq    10.2G
storage/Debian-0l4yn4      9.69G
storage/NASLoup            4.18T
storage/Ombi-qio5l6        9.69G
storage/Ombi-zr2xil        9.69G
storage/iocage             96.6G
storage/webDB               128K
root@freenas[/mnt/storage]# du -sh *
7.7T    iocage
1.7T    NASLoup
512B    webDB
 
Top