Getting best performance with 7 HDDs?

[ysnk]

I just did my first test install of Freenas - everything worked perfectly. I would appreciate if someone can advice on the best way to setup drives in my case:
Main usage - video fx /editing, so I need a fast read/writes of large files and image sequences. I'm getting 10gb nic in a couple of days. I have only 6 computers in my office with 3-4 people working at the same time.
What would be the best way to setup HDDs for max performance?
Will raidz2 work file or should I use something else?
I have 7 hdds now - 4 WD RE4, 2 Hitachi 7k3000, and Seagate Nas drive. All 2tb. By the way, is that okay to mix all of those?
Thanks.

 

[dlavigne]

Which solution did you end up implementing?

 

[danb35]

Do you just want highest performance, or are there other factors as well? If you want the highest possible performance, just stripe the disks. That, of course, gives you no redundancy, so when one disk fails you'll lose your data. This also gives you the highest possible capacity. The next highest performance solution would be stripped mirrors of an even number of disks. You give up half of your capacity to redundancy this way, but it's still pretty quick. Any RAIDZ configuration will be considerably slower.

 

[ysnk]

I bought one more HDD (8 total), decided to go with stripe of 4 mirrors. Waiting for it to arrive, as well as ibm HBA.
Running stripe of 3 mirrors now. With 10Gb connection I'm getting about 450MB/s write, 300MB/s read then I'm copying large files, but it drops to about 200 write / 150 read for the folder with mixed small files. I'm using Windows share. Is that performance normal? Seems low...
I have only 8GB of ram now, ordered 8 gig more.

 

[Ericloewe]

More RAM will definitely help, especially with lots of small files.

Honestly, it's rather weird to see 10GbE in a system with only 8GB of RAM.

 

[ysnk]

I was planning to get 16gb ram from the start but ordered a wrong one - turned out mb works with the unbuffered ram only.
But will it really help improve read speed? Write is 30% faster then read now, is that normal?

 

[sfcredfox]

I was planning to get 16gb ram from the start but ordered a wrong one - turned out mb works with the unbuffered ram only.
But will it really help improve read speed? Write is 30% faster then read now, is that normal?

I won't offer this as an expert opinion or an answer, but I see a lot of posts that mention the read speeds of their drives are slightly slower than writes. It's known that more vdevs produce better read speeds. If ignoring cache (ARC) and just talking about raw device performance, my guess for better read performance is more vdevs or faster drives.

Maybe an expert level guy can confirm or dispute this?

 

[ysnk]

Makes sense.
I'll post an update next week on how additional 8Gb and extra mirror vdev will affects the performance.

 

[Ericloewe]

Writes being faster is unusual, but I wouldn't read too much into it without a properly controlled test.

 

[sfcredfox]

Writes being faster is unusual, but I wouldn't read too much into it without a properly controlled test.

By saying it's unusual, you're saying on a proper configuration, one should expect their read to be better then their writes?
Is it a fair statement then to say if you see writes out performing your reads, you might have a configuration or hardware issue? Like he as asked?

 

[Ericloewe]

By saying it's unusual, you're saying on a proper configuration, one should expect their read to be better then their writes?
Is it a fair statement then to say if you see writes out performing your reads, you might have a configuration or hardware issue? Like he as asked?

Not exactly. I can imagine scenarios where this would happen, but vary the workload and the picture immediately changes...
Reads, hardware-wise, are typically faster than writes. That's what makes it unusual - but it can be the workload tested that's causing the abnormality.

 

[vikingboy]

I suspected in cases like this writes may be affected by being reported as written into ram where as reads have to come from disk?

 

[depasseg]

Could the Windows system at the other end of the transfer test be impacting the performance?

 

[Ericloewe]

Could the Windows system at the other end of the transfer test be impacting the performance?

There are just so many possibilities I can think of, plus many more I can't, that it doesn't mean much without proper context.
It could be the mish mash of drives, an odd workload, network loading, network capacity...

 

[cyberjock]

I guess I'll pipe in as the "expert".

Write speed vs read speed is a red-herring. It really doesn't mean anything unless there's a massive disparity (think 100MB/sec write and 1GB/sec read). Even then, you have to know whether those speeds are normal or not. Writes are generally cached, which means until you fill the write cache completely, the claimed speed is NOT your pool. I can write almost 3GB in 3 seconds over my 10Gb. But once those 3GB are in the write cache performance drops rapidly to a more reasonable number as the pool is having to handle all the writing and can't actually do 1GB/sec writes. Likewise, if I am reading the same file from the ARC over and over, I can get nearly 10Gb LAN speed despite the fact that my pool can't keep up with that workload. Then things like pool fragmentation and sync writes can also impact performance. There's waaaay too many variables and unless you are going to contract someone to actually sit down and analyze your server it's best to just take your speeds as they are. ;)

So no, comparing write and read speeds don't tell you much of anything except that "one is faster than the other".

I will say your numbers seems to pretty much align with what I'd expect for that hardware. If you want more performance add more RAM first. 8GB of RAM is the minimum, and it makes no promises whatsoever of having even "good" performance.