AV1 Video encoding with 8 simultaneous threads. Local HDD vs SMB 12 disk vdev raidz2?

[Videodrome]

Let's say I have a 1.5TB file (Apple prores), that I want to encode to AV1 (Av1an with 8 workers).
These 8 threads will be reading the 1.5TB file simultaneously in small random chunks and write the output to another local disk.
AV1 encoding is pretty slow, even with multiple threads (maybe 20fps in my case), so this scenario is not about max. sequential performance, and more about IO and if SMB is as fast as a local ahci sata hdd.

What would be faster?
Having the 1.5TB file on a 12 drive raidz2 (1mb blocksize, no sync), and reading it through SMB with a 1gbit nic?
or
Copying the 1.5TB locally to a single SATA HDD drive?

And,
Would a 10gbe nic make any difference? (I'd be surprised..)


Thank you.

 

[sretalla]

What would be faster?
Having the 1.5TB file on a 12 drive raidz2 (1mb blocksize, no sync), and reading it through SMB with a 1gbit nic?

no, 1Gbit is not faster than a single local HDD (assuming that single drive can do about 150MB/s reads). The best you can expect from a 1Gbit NIC is about 125MB/s.

Would a 10gbe nic make any difference? (I'd be surprised..)

Indeed you will be surprised then... 10Gbit can do 1250MB/s, so far in excess of a single local HDD.

Provided your pool is connected properly and you have sufficient RAM, you will certainly get more than 150MB/s over 10Gbit.

 

[Videodrome]

This is not about max. output.
It is about 8 threads to the same very large file, and which scenario can serve random chunks of data faster to encode.

The combined data rate of those 8 threads will probably not exceed 30mb/s.

My gut tells me a single local sata hdd would be faster since it doesn't have to go over an smb connection.
Ideally, the storage server can handle this just as fine.

I'll do some testing myself.

 

[ChrisRJ]

This is not about max. output.

Well, that is what you asked for in the initial mail.

It is about 8 threads to the same very large file, and which scenario can serve random chunks of data faster to encode.

That is still about the max. speed.

The combined data rate of those 8 threads will probably not exceed 30mb/s.

As a mostly sequential workload that should be faster locally, as @sretalla already mentioned.

My gut tells me a single local sata hdd would be faster since it doesn't have to go over an smb connection.

When made as such a general statement that is wrong. Besides, faster is not only about bandwidth but also latency. So there are workloads where a NAS with SSDs and 1 Gbps NIC will be faster than a local HDD that can do 200 MB/s sequentially.

I'll do some testing myself.

That is always a good idea. Esp. since you told us nothing about your hardware (NAS as well as client).

 

[sretalla]

A single SATA disk won't fetch 8 separate streams faster than a ZFS pool.

 

[Videodrome]

It seems people are unwilling to understand the problem or have reading comprehension issues.
Sorry to sound harsh, but I was very clear about my problem.

This was and has never been about max. output of sequential data.


Anyway.
I did some testing.

It seems that NVME, HDD, and a 1gbit to a storage server are nearly the same in encoding speed.
Only the initial 'scene-detection' phase was a lot slower on the 1gbit (about 33% slower).
Which seems logical since that is the only time where it needs to scan the whole file as fast as possible.

This thread can be closed.