New build, problematic performance

[Minxster]

Swapping out the HBA certainly is an option. I know the mpr driver is newer than the mps one so it might have some unresolved bug that you're bumping into.

Thats kind of my thought TBH, I've rechecked recently and the client has actually re-upgraded the card to 100% it's as up-to-date as possible. I just can't let go that it's not the CPU because of the testing/results thus far, especially the DD testing.

Hopefully we'll get some more evidence via the on-board controller soon.

 

[Minxster]

So INTEL Xeon GOLD 6128 3.4GHz has now been fitted to the rig. We re-ran performance tests with DD and real-world. We still just couldn't get 2 x RaidZ2 (8 disks each) to work for the real-world.

I searched around for posts to do with mirrored array speeds and why the DD testing just seemed really odd!I came across this:
https://forums.freebsd.org/threads/zfs-read-performance-of-mirrored-vdevs.59879/

The (above) forum posts are really interesting, for me they seem to indicate that ZFS and mirrors must have some issues. But how that translates into data being processed vs. disk status I'm not too sure about. I still don't really get why writes are quicker than reads though

Then, luckily for me, the client ran real-world testing on the, what I'll now call, a bunch of mirrors (8 2-disk mirrors). Though we still got poor read speeds, it actually seemed to work correctly finally getting some consistent data transfers with good IOPs.

There was a lot of overhead going on with SMB, so ended up using iSCSI instead. We're now, generally speaking, getting around 900Mbytes writes and over 1Gbyte reads per second. And with the IOPs of running mirrors it's chewing through DNG files like they're going out of fashion! I made sure that the record size was set to 1M, and during testing did show a performance increase that was significant.

I've still got a lot to learn ;), but why we never real-world tested the mirrors, I've got no idea. I think the fixation of large transfer speeds always pointed to the RaidZ2 setup, but really we could not get enough IOPs for the DNG files to work correctly with the video editor.

Did we really need the high end CPU? I don't think we'll really find out as the previous processor won't be going back into the rig. So will it FreeNAS? Yes, but pay extra care into IOPs and make sure you do your DD testing and real-world testing!

 

[AndroGen]

Hi @Minxster,
may I ask you for some clarification behind the final solution?
Background is, I am evaluating options for my FreeNAS and one of them is a similar Matherboard: Supermicro X11SPM-F.
What I could not figure out from this conversation, whether you still use the disc controller from the motherboard, or had to install additional card for this?
Or the problem was only related to the way how all disks were configured in the pool?
If it was a card, which one had to be added, was it NIC?

 

[Bozon]

... The solnet-array test did not 100% complete due to the hotel wifi I was using failing while I was remotely using putty. ....?

Use nohup ( no hang up ) in front of the command so that the command will continue even if you are booted off your connection. Or if you are already running the command ctrl-z and then use the disown command, see full description below:

1. disown -h [job-spec] where [job-spec] is the job number (like %1 for the first running job; find about your number with the jobs command) so that the job isn't killed when the terminal closes.

 

[Bozon]

This guide has the steps for creating a resumeable session so you don't loose the test because you loose connectivity

Uncle Fester's Basic FreeNAS Configuration Guide
https://www.familybrown.org/dokuwiki/doku.php?id=fester:intro

Yes screen is the bomb. I couldn't see doing remote system admin without it. For simpler, purposes nohup and disown -h are useful commands.

 

[Bozon]

Interestingly, disown doesn't have a man page on my FreeNAS 11.2 server, but it seems to work. nohup has a man page and seems to be what I expect. I forget that Linux and BSD have some differences, and I need to check. I am really used to the way ps works in Linux, but the options are completely different in BSD.

 

[Bozon]

Interestingly, disown doesn't have a man page on my FreeNAS 11.2 server, but it seems to work. nohup has a man page and seems to be what I expect. I forget that Linux and BSD have some differences, and I need to check. I am really used to the way ps works in Linux, but the options are completely different in BSD.

Disown is a bash built-in, so it may not have a separate man page.