Not getting expected speeds on all SSD pool

mmorabito606

Cadet
Joined
Sep 9, 2020
Messages
1
Hello,

I've been doing performance testing on my SSD ZFS pool and have been banging my head up against a wall for a few weeks now trying to get the expected performance out of it.

Please note that the NAS that I'm performing these tests on is NOT in production. This configuration described here is solely for performance testing reasons.

My specs of my NAS are as follows:

Server: Dell R730xd
RAM: 32GB DDR4
CPU: 2x Intel Xeon CPU E5-2620 v3 @ 2.40GHz
RAID: PERC HBA330
Drives: 2x Generic SSD 120GB (for FreeNAS boot), 10x Samsung MZILS3T8HMLH0D3 3.84TB SAS 12Gbps (for ZFS pool)
Additional hardware: Intel Optane 900p 280GB (for ZIL or L2ARC if needed)

My assumptions of ZFS/FreeNAS are based on the article below so please correct me if I have anything wrong:

Before all my doubt started I was getting seriously slow speeds for these SSDs. The server needed some power options in the BIOS adjusted before it started giving me decent speeds which can be found here:

As my setup currently stands, I have a single 10-drive stripped VDEV in a pool with a virtual image (running through Bhyve on FreeNAS, local to the ZFS pool) running a dd command achieving 850MB/s write. The dd command I'm using is as follows:

dd if=/dev/zero of=test bs=1G count=1 oflag=dsync status=progress
io-test.PNG


Are these results sufficient or am I expecting too much out of ZFS? I feel like I see people reaching numbers like these in 4-drive RAIDZ1 configurations with spinning rust so I'm kind of weary of these results.

The Optane mentioned above is not being used at the moment as I was reading that a ZIL is not really necessary in an all SSD pool. That didn't stop me from trying anyways but it seemed like the advice I read was true as I didn't see any real performance gain/loss from enabling it.

I did disable sync which got me up to 1.0GB/s but that is far from what I expected.

From searching around I found that a single one of my SSDs should be performing around 1250MB/s read and 1100MB/s write. Would I be a fool to think if I created a stripped VDEV of all 10 drives then I would ideally get 11GB/s write and 12.5GB/s read?

Also, is there any way to test the performance of each individual drive before creating a VDEV/pool out of them?
 
Top