TrueNAS virtualized in ESX gets exhausted

[bonne]

Running TrueNAS-13.0-U6 inside ESX 7 - four ST2000DM008 HDDs passthru onboard SATA controller (Supermicro)

VMXNET3 NICs, 2 CPUs 16 GB RAM.

One storage pool RAIDZ2, no snapshots, lz4 compression, no dedupe presented to ESX as iSCSI

Disksystem seen as SSD to ESX and the VMs (is this an issue when the actual disks are HDDs?)

It works, but I would expect better performance - especially after running heavily, the disksystem gets tired.

Funny enough, running crystaldiskmark, seauential operations seems very fine... random seek not... attached is a run with default settings.

But real life operations suck. If I start to copy a 3 Gig file, Windows reports around 60MB/s, but as times goes, it falls down to 10 MB/s

What am I doing wrong?

 

[Patrick M. Hausen]

four ST2000DM008 HDDs passthru onboard SATA controller (Supermicro)

Did you pass through the entire controller as a PCIe device or did you pass through the disks? The latter is not a recommended working configuration.

 

[HoneyBadger]

four ST2000DM008 HDDs

These are Shingled Magnetic Recording (SMR) drives and are likely a significant contributing factor.

 

[Etorix]

One storage pool RAIDZ2, no snapshots, lz4 compression, no dedupe presented to ESX as iSCSI

And raidz2 is not recommended for iSCSI.

Resource - The path to success for block storage

ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle bits modified after creation. This is optimal for RAIDZ. It's what most...

www.truenas.com

Re-do your pool as (stripped) mirror of CMR drives.

 

[bonne]

Did you pass through the entire controller as a PCIe device or did you pass through the disks? The latter is not a recommended working configuration.

The controller

 

[bonne]

And raidz2 is not recommended for iSCSI.

Resource - The path to success for block storage

ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle bits modified after creation. This is optimal for RAIDZ. It's what most...

www.truenas.com

Re-do your pool as (stripped) mirror of CMR drives.

Ok - would NFS be better, or same as iSCSI?

RAID10 - and which disks would be a good choice for this of today? Larger SSD disks seems to have come down to a reasonable price. Maybe instead I should go for four say Samsung EVO SSD's - will they work nicely in such a TrueNAS setup?