What performance to expect?

[Johanna12221]

Hello,
I'm about to set up TrueNAS SCALE and I've been trying to find some real life performance tests of what to expect.

I'm looking into running a 2 disk redundancy, so I'm mostly interested in these cases:

- RAIDz2: 8 x 20TB, WD Ultrastar DC HC560 SATA (291 mb/s max, Datasheet)
- RAIDz2: 8 x 18TB, WD Ultrastar DC HC560 SATA (269 mb/s max, Datasheet)
These would be run on:
- Intel Xeon 2620 v3
- 32 GB ECC
- LSI 9300-8i HBA

I understand it's impossible to say exactly, but around what could be expected read/write speeds?

 

[ChrisRJ]

You left out the factor that has the biggest impact: the workload. It makes an extreme difference (several orders of magnitude) whether we are talking sequential vs. random, read vs. write, sync vs. async writes etc.

What is your requirement/use-case?

Please check the forum rules (link in red at the top middle screen) on the information you should provide to get the best possible answer.

Lastly, please check the recommended readings from my signature to get additional information.

 

[Johanna12221]

Hi, I understand it depends widely on the usage, but I'm looking into some generic read/write speeds that one could expect.
Like are we talking 100/500/1000/mb/s, some sort of pointer.

I'm hearing that hardware RAID is faster on read, but slower on random for example, and since I have a hardware controller that could also be an option.

The storage will mainly be used for archiving of media files. Some portion of the storage (datapool?) will be used for some streaming of music files and regular document/photos etc.

 

[ChrisRJ]

Like I said, there are orders of magnitude in the game. You could have 500 MB/s for some workload and 5 MB/s for another.

Hardware RAID is not an option with ZFS, see recommended readings.

The scenarios you mention are not very demanding and similar to mine. There is still some details missing on the hardware and it would help to know the motherboard.

 

[Johanna12221]

Thanks for your reply,
The motherboard is a Z10PE-D16 WS. What parts of the motherboard will affect the performance in positive/negative way? Like PCI lanes etc?

When I talk about hardware RAID I mean keep running hardware RAID completely, with a 9460-16i that I'm planning to migrate away from.

This is the performance I get as of now, over a Samba share.
It's the random and the fact that sequential write is higher than read that I can't get my head around.

This test was run on the 8 TB Ultrastar with "Record size" set to 256kb, RAIDz2.
Is it reasonable?

 

[jgreco]

Hi, I understand it depends widely on the usage, but I'm looking into some generic read/write speeds that one could expect.
Like are we talking 100/500/1000/mb/s, some sort of pointer.

What's "mb/s"? Megabits per second? Megabytes per second? Since this is a technology forum, it's necessary to disambiguate oneself when posting. Please see

Terminology and Abbreviations Primer

We realize that new users have a lot to learn when they come to FreeNAS. There's a certain amount of confusion added to discussions when users pick random/approximate terms to describe things. I've spent a lot of time quietly trying to translate terms on the reader's side when reading posts...

www.truenas.com

I'm hearing that hardware RAID is faster on read, but slower on random for example, and since I have a hardware controller that could also be an option.

Where did you hear that? Certainly not here. Hardware RAID is fundamentally *incompatible* with TrueNAS. See

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

The tiny sucky CPU's on a RAID controller and the itty bitty amounts of cache that they might happen to have can kill your performance, that is, right up until something happens where the RAID controller barfs and messes up your virtual disks.

the fact that sequential write is higher than read that I can't get my head around.

Why? It makes perfect sense. When you're writing, you are writing to the ZFS "write cache", which is actually called a "transaction group" or txg. This is in your system's main memory, and under optimal conditions, which you are probably close to if the system is relatively new, writes come in from the network, placed in memory, and then in a few seconds are staged out to disk, while a new txg is opened and begins filling. This effectively means that writes happen as fast as ZFS can write your stuff to memory. Reads, on the other hand, have to be fulfilled from disk, by going out to the pool, seeking, reading the data in from disk, and then copying that back to the client over the network. This can still happen pretty fast for sequential data, but seeks will impact the read speeds possible.

 

[Johanna12221]

To clarify: When I'm talking about hardware raid it has nothing to do with TrueNAS, it's solution I'm looking to migrate away from going to software raid with TrueNAS and a HBA card instead.

I will make sure to clarify better when it comes to speeds, so to clarify now, we can talk about MB (megabyte).

Thanks for explaning why write is faster than read. Would you also say that the random read/write are what to be expected?

When I'm using the same disks but connecting them to my hardware RAID (9460-16i vs 9300-8i HBA) I get around 1000MB/s sequential read speeds and I was actually thinking every speed test would perform better with TrueNAS

 

[ChrisRJ]

Unless I overlooked something, you did not specify in detail how the performance test was done. To understand where the difference comes from, this information is needed.