Storage Pool optimization, e.g. fusion pool, for SMB

[DigitalMinimalist]

Dear all,

the main application as storage server of my future TrueNAS Scale Server will be via SMB to supply media to mainly Windows PCs, Media Center, e.g. Nvidia Shield and as backup location from Windows PCs.

Complete Hardware list:
Asrock Rack X470D4U
Ryzen 2700X
Noctua NH-U9DX i4
4x16GB ECC DDR4 Samsung 2666Mhz -> 64GB RAM
Crucial P2 250GB for TrueNAS Scale OS
One additional NVME slot available
Asus Hyper V2 with 4x Micron 7300 Pro 960GB (PLP, 1.9PB TBW)

• 2x mirrored for VMs
• 2x mirrored for special vDev?

5x 3.5 Backplane RaidSonic Icy Box IB-555SSK

• initially 4x HDD 4TB CMR for performance testing: striped mirror or RAIDZ2 -> upgrade later to 16TB drives

Intel X710-DA2 NIC
Seasonic Prime TX-1000
UPC: APC Smart-UPS X 1000VA

Though I'm trying to explore the best vDev / Disk setup for SMB usage with the mentioned hardware.
I was originally intending to add SLOG and L2ARC, but then read a lot and realized that more RAM and special vDev for Metadata might be a better option

I'm still exploring and testing the different Pool Designs to find the best setup - are there any other configs which you could recommend to explore or test?
Are there any other "tuning" options for fusion pools?

Current options:

1. RAIDZ2 of 4x HDD
2. Striped Mirror of 2x 2 HDD
3. RAIDZ2 with NVME mirror for special vDev/Metadata
4. Striped Mirror with NVME mirror for special vDev/Metadata
5. ???

Looking forward to your experiences, especially fusion pools with SMB

Best
Alexander

 

[spuky]

ZFS Metadata Special Device: Z

Introduction ZFS Allocation Classes: It isn’t storage tiers or caching, but gosh darn it, you can really REALLY speed up your zfs pool. From the manual: Special Allocation Class The allocations in the special class are dedicated to specific block types. By default this includes all metadata...

forum.level1techs.com

might give you some insight in the benefits and math arround fusion pools

the post is not truenas or freenas specific... but all options that are used are also exposed in the UI

 

[ChrisRJ]

What does your workload look like?

For ZFS background (incl. vdev design) please check the recommended readings in my signature.

 

[DigitalMinimalist]

might give you some insight in the benefits and math arround fusion pools

Thanks - I saw the L1T post a while ago, but forgot - thank you

What does your workload look like?

NVME Mirror for IOPS critical applications: the VMs should be fine
Plex, or Emby
Omada, or Unifi Controller
Nextcloud
pfSense, or OPNSense

Storage Workload - faster Pool has the only advantage to quicker upload / download - IOPS should not be critical for this pool
Central Media Storage of Movies & Music
Central Backup location of large Photo Library - probably RClone to Cloud
Central Storage of Downloads
Central Storage of other backups, e.g. smartphones

For ZFS background (incl. vdev design) please check the recommended readings in my signature.

Any specific guide to read through in detail?

 

[DigitalMinimalist]

To add:

We are talking about home usage of non critical data.
I will therefore disable sync

Is there a max speed via SMB? I see peak transfer rates in the area of 7Gbps and average in the area of 5.
Is it even possible to get into the >9 Gbps area with SMB?

SMB3.0 does not seem to be supported yet by Scale.

I saw some posts how to adjust the size threshold and need to fiddle around