Which setup for 16 SSDs?

[DirkTripleD]

Hi all,

I bought 16 KINGSTON 3.84TB DC450R SEDC450R/3840G for a customer system for 4K video post-production with Adobe Premiere.

Which will be the best setup?

Should I use two RAID-Z1-pools of 8 drives each?

Or one pool with RAID-Z2 out of all 16 drives?

I am using two HBAs (one onboard, it is an Supermicro Motherboard X11DPi-NT), each is able to handle 8 drives.

The additional NIC is an Intel-XXV710 based one with 25G Dual-Port SFP28.

Additionally I bought two NVMe SSDs with 2TB capacity for L2ARC (on a Supermicro Add-on Card AOC-SLG3-2M2 riser-card, I hope, TrueNAS is able to handle it).

The CPU is an INTEL Xeon Scalable 4216 2,10GHZ, the ECC-RAM is 256 GB.

I found this interesting summary but I am still unsure:

How to Improve ZFS performance

First Edition: March 14, 2010 Last Edited: January 18, 2021 In this tutorial, I will show you how to improve the performance of your ZFS using the affordable consumer-grade hardware (e.g., G…

icesquare.com

Thanks in advance!

 

[NugentS]

How long is a piece of string.
What is best depends on what you are trying to achieve
Personally I think the best solution is to send them all to me!!

 

[DirkTripleD]

I would like to receive enough speed for 6 via 10 GbE connected Lenovo P620-workstations and some security against drive failure (but this is not really necessary, because it is not for storage purposes but for editing only).

As I personally have never seen a dying SSD right now, probably RAID-Z1 would be fine, but in the mentioned link are stated some advantages in speed of RAID-Z2 if the pool is very full.

It would be nice if someone with similar needs and a proven configuration could answer. Thanks.

 

[NugentS]

If you want speed use mirrors
If you want capacity use Z1,2,3

Sean

 

[NugentS]

If you want live access, ultra high speed then you should be using mirrors. There was a really good article by I think arstechnica but I can't find it atm.
16*3.8/2 = 25+TB or so. Is that enough as that will be as fast as you can go.
I am assuming your HBA's can handle that - not many can (I mean in IOPS - not TB capacity)

https://arstechnica.com/gadgets/202...ht-ironwolf-disks-two-filesystems-one-winner/

And of course you don't tell us what HBA's you are using / planning on using. Oh BTW that motherboard, according to the specs I read does not have an HBA although it does have a boatload of SATA Ports so you might be using those.

Really - you are asking us to guess

Spec please - properly

 

[DirkTripleD]

Sorry, I forgot to mention, the second 'HBA' is a LSI MegaRAID SAS 9361-8i 8-Port 12G SAS/SATA RAID Controller in IT-mode.

I set it up with Z1.

The system is sold and working great.

 

[Etorix]

Video post-production means you want high throughput for temporary files: This requires as many vdevs as possible, mirrors would be best. You may even consider a stripe of single drives if this only for scratch files. 8 * 2-way mirrors should serve you well.
If the NAS also stores the rushes, to be kept safely during production, and/or the final files, to be kept indefinitely, HDDs in raidZ2 or raidZ3 would be better suited—SSD are overkill for mere long time storage.
You do not have enough memory for 4 TB of L2ARC, and I do not see any point for a L2ARC if your data is already on fast SSD storage.

 

[Constantin]

Persistent, metadata-only L2ARC on SSDs makes a lot of sense for much faster browsing of largely dormant HDD pools. you need CPU memory to spare for the L2ARC index (32GB and up is often cited as a suggested minimum).

Some protocols / programs also traverse directories a lot, like rsync and rsync-based backup programs.

As of TrueNAS 12, another similar option is sVDEVs, which require far more careful thought upfront re installation to maintain pool health and so on.

but in your case, I doubt much can be done with a L2ARC when the pool is already SSD. More CPU memory likely has the biggest positive impact then.