Where is a VM disk file actually physically located in the pool. Is it hidden?

[titust1]

I created a few VMs and Scale automatically created a Z-Vol for each VM. A Z-vol is in fact a dataset block device located on the Z-Pool.
But I guess it should be a file. I tried to locate the file using the shell as root, but I don't see any files in the Z-Pool dedicated to VMs.
Where is the VM disk actually physically located? Is it hidden somehow?
Thanks

 

[Patrick M. Hausen]

I created a few VMs and Scale automatically created a Z-Vol for each VM. A Z-vol is in fact a dataset block device located on the Z-Pool.

Correct.

But I guess it should be a file. I tried to locate the file using the shell as root, but I don't see any files in the Z-Pool dedicated to VMs.
Where is the VM disk actually physically located? Is it hidden somehow?

You wrote a zvol is a block device stored on a zpool. Which is correct. Hence, a zvol is not a file. Why do you "guess it should be"? It's not a file and never will be. It's a block device.

You can list your zvols with zfs list -t volume, you can snapshot them with zfs snap, you can replicate them, you can view them in the UI under the Storage menu ... still not files.

 

[Samuel Tai]

I believe zvols reside in /dev/zvols/<pool name>/<path to zvol in pool>/<name of zvol> as a character block device, not a file. They're not like a VMDK or other virtual disk format.

 

[titust1]

Right, thanks

Code:

root@diskstation# cd /dev/zvol/Z-Pool-VMs
root@diskstation[/dev/zvol/Z-Pool-VMs]# ls -ails
total 0
793 0 drwxr-xr-x 2 root root 60 May 12 19:47 .
792 0 drwxr-xr-x 3 root root 60 May 12 19:47 ..
794 0 lrwxrwxrwx 1 root root  9 May 13 14:02 Win10-wa1c3c -> ../../zd0

 

[sretalla]

I'm curious to know what you think knowing that location will help with...

 

[titust1]

I'm curious to know what you think knowing that location will help with...

Out of simple curiosity.
I think that using for years at work Vmware, then Hyper-v, and at home recently playing with Proxmox, I got used to deal with VMs as files

 

[titust1]

I'm curious to know what you think knowing that location will help with...

I also wanted to have an idea about the disk space used by a zvol, and why I have such a low space. By listing the z volume, I can see:

Code:

root@diskstation[~]# zfs list -t volume
NAME                      USED  AVAIL     REFER  MOUNTPOINT
Z-Pool-VMs/Win10-wa1c3c   185G   203G     18.6G  -

I still don't understand a many things.
My Z-pool dedicated for the VMs is on a 256GB drive.
I've created just one VM with a 164GB drive (because of the 80% rule).
The Windows VM, occupies just 30GB, but still I have an warning that says Z-pool is occupied at 85%
I don't get it. Maybe you do understand....

 

[sretalla]

Snapshots?

zfs list -t snap

Also you should look up the difference between GiB and GB.

 

[titust1]

difference between GiB and GB.

Don't worry yourself about that, I'm a retired IT guy.
And no I don't have snapshots yet

 

[Samuel Tai]

Looking at file system "usage" isn't helpful, because ZFS's on-disk format is usually too indirect for simple utilities like ls to deal with. You should instead look at zfs get used <zpool path to dataset> to see the actual on-disk usage. This will also be displayed in Storage->Pools in the UI.

For more insight, also examine the ZFS properties usedbysnapshots, usedbydataset, usedbychildren, and usedbyrefreservation per dataset.

Also, remember ZFS is a CoW (copy on write) filesystem. The initial zvol volume consumed may expand quite quickly, depending on how much your VM writes to the zvol. Only when the VM shuts down, can ZFS do housekeeping and mark blocks superseded by new writes as unreferenced within the zvol, and not count them towards the used figure.

 

[Samuel Tai]

root@diskstation[~]# zfs list -t volume
NAME USED AVAIL REFER MOUNTPOINT
Z-Pool-VMs/Win10-wa1c3c 185G 203G 18.6G -

OK, you created a static zvol using 80% of your pool. Inside the VM, it only shows 30G in use. It would've been better if you had created a 60G zvol instead.

For example, here's a 25GiB zvol I set up for one of my Linux VMs, GPT-partitioned into a 4GiB swap partition and a 20GiB ext4 partition:

Code:

root@raven:~ # zfs get all main/zvols/rancher-zvol
NAME                     PROPERTY                 VALUE                    SOURCE
main/zvols/rancher-zvol  type                     volume                   -
main/zvols/rancher-zvol  creation                 Thu Mar 26 18:24 2020    -
main/zvols/rancher-zvol  used                     52.0G                    -
main/zvols/rancher-zvol  available                4.37T                    -
main/zvols/rancher-zvol  referenced               21.2G                    -
main/zvols/rancher-zvol  compressratio            1.24x                    -
main/zvols/rancher-zvol  reservation              none                     default
main/zvols/rancher-zvol  volsize                  25.0G                    local
main/zvols/rancher-zvol  volblocksize             16K                      -
main/zvols/rancher-zvol  checksum                 on                       default
main/zvols/rancher-zvol  compression              lz4                      local
main/zvols/rancher-zvol  readonly                 off                      default
main/zvols/rancher-zvol  createtxg                14849267                 -
main/zvols/rancher-zvol  copies                   1                        default
main/zvols/rancher-zvol  refreservation           25.4G                    local
main/zvols/rancher-zvol  guid                     13039343090551318795     -
main/zvols/rancher-zvol  usedbysnapshots          5.47G                    -
main/zvols/rancher-zvol  usedbydataset            21.2G                    -
main/zvols/rancher-zvol  usedbychildren           0B                       -
main/zvols/rancher-zvol  usedbyrefreservation     25.3G                    -
main/zvols/rancher-zvol  dedup                    off                      local
main/zvols/rancher-zvol  refcompressratio         1.26x                    -
main/zvols/rancher-zvol  written                  66.7M                    -
main/zvols/rancher-zvol  logicalused              29.2G                    -
main/zvols/rancher-zvol  logicalreferenced        23.6G                    -
main/zvols/rancher-zvol  volmode                  default                  default

You can see the 25G definition under the volsize property. The actual space consumed is 52G, according to the used property. This is divided over the 5.47G used for snapshots, 21.2G used by the VM within the zvol, and 25.3G used to track block reference reservations for written blocks superceding earlier versions of those blocks. Note, all units are GiB.