Are virtual machine zvols created from the GUI optimized for performance?

[Intel]

Reading some ZFS optimization guides they recommend to use recordsize/volblocksize = 4K and disable compression (https://www.medo64.com/2019/09/zfs-pool-for-virtual-machines/).

Looks like by default this is what is getting set:

Code:

root@nas[/mnt/nxg]# zfs get all nxg | grep recordsize
nxg   recordsize            128K                   default
root@nas[/mnt/nxg]# zfs get all nxg/test/win10-ssktgv  | grep size
nxg/test/win10-ssktgv  volsize                40G                    local
nxg/test/win10-ssktgv  volblocksize           16K                    -
root@nas[/mnt/nxg]# zfs get all nxg/test/win10-ssktgv  | grep comp
nxg/test/win10-ssktgv  compressratio          1.27x                  -
nxg/test/win10-ssktgv  compression            lz4                    inherited from nxg
nxg/test/win10-ssktgv  refcompressratio       1.27x                  -

Has anyone experimented changing these settings and observed noticeable improvements? are these truenas ZFS defaults on VM zvols appropriate and optimized already?

 

[morganL]

4K is an optimization for database IOPS.... many workloads will prefer a larger blocksize.
What is your workload?

 

[Patrick M. Hausen]

If you run a VM with Ext4 or NTFS, both having a 4k native block size, wouldn't it be best to use a ZVOL with an identical block size for the virtual disk? I have been doing this since I started using VMs, but never ran any benchmarks.

It doesn't matter what the workload is - Ext4 will always write 4k chunks. As will NTFS.

 

[Intel]

4K is an optimization for database IOPS.... many workloads will prefer a larger blocksize.
What is your workload?

I run several Windows 10/11/Server VMs for experiments. Sometimes I also run linux VMs.

If you run a VM with Ext4 or NTFS, both having a 4k native block size, wouldn't it be best to use a ZVOL with an identical block size for the virtual disk?

Yeah this is apparently what is recommended in multiple places as I was reading up on this - 4K block size is not used by TrueNAS Scale when it creates the zvol for a new VM; I can do this manually and maybe someone else here does it already?

The default settings used by TrueNAS seem 16K and I am curious why it uses that... maybe TrueNAS team has done benchmarks with multiple settings and 16K is better than 4K?

 

[Patrick M. Hausen]

16k is simply the default blocksize for ZVOLs as 128k is for datasets and most probably nobody gave a thought to making that configurable in the UI or changing it at all.

 

[morganL]

If you run a VM with Ext4 or NTFS, both having a 4k native block size, wouldn't it be best to use a ZVOL with an identical block size for the virtual disk? I have been doing this since I started using VMs, but never ran any benchmarks.

It doesn't matter what the workload is - Ext4 will always write 4k chunks. As will NTFS.

Being more precise, they will write in multiples of 4K.
Extents can be very large. https://en.wikipedia.org/wiki/Ext4#:~:text=An extent is a range,a 4 KiB block size.

So the workload does have an impact.
16K is common for database workloads... MySQL, SQL server.
Oracle can run at 4K
VMware (VDI) is typically around 24 - 32K but that is accessed via iSCSI/FC

 

[fahadshery]

Being more precise, they will write in multiples of 4K.
Extents can be very large. https://en.wikipedia.org/wiki/Ext4#:~:text=An extent is a range,a 4 KiB block size.

So the workload does have an impact.
16K is common for database workloads... MySQL, SQL server.
Oracle can run at 4K
VMware (VDI) is typically around 24 - 32K but that is accessed via iSCSI/FC

what about what about proxmox?

 

[fahadshery]

I am indeed passing the ZVOL as iSCSI to proxmox so that I could run VMs on it.

 

[fahadshery]

I was using the default 16K volblocksize. but if that needs changing for performance gains, I am up for it

 

[Patrick M. Hausen]

You will have to check proxmox documentation and/or community for an optimal block size, I guess.

 

[morganL]

You will have to check proxmox documentation and/or community for an optimal block size, I guess.

It will depend on the type of VMs running on Proxmox... database VMs may need smaller block size than image processing VMs.