Stutter in CIFS performance

[detyrious]

Hi there,

I'm relatively new to the whole NAS thing, so bare with me.
I built myself a NAS box to use as a home server, i.e. fileserver, media server, backup.

Hardware:
MB/CPU: ASRock E350M1/USB3 with AMD E-350 CPU (2x1,8GHz)
RAM: 8GB Kingston ValueRAM
DISKS: 4x 2000GB Seagate Barracuda Green 5900.3

At first I only had three disks and everything was pretty smooth. Performance with CIFS was stable and around 70MB/s.
But yesterday I added the fourth disk, destroyed the ZFS Pool and created a new one (RAID-Z). Since then my CIFS performance is pretty much broken.
If I try to write a large file (~12GiB) to a CIFS share, the speed starts at 100MB/s, goes down to around 70MB/s and then stops completely for a couple of seconds. After it picks up again the speed floats around 30-40MB/s and completely stops every few seconds.

In Bandwith Monitor it looks something like this: http://goo.gl/8EZqI

Is there any way to improve performance?
Even if I try FTP oder NFS the same problem occours.
So...

a) Does the CPU have not enough power for a four disk raid-z?
b) Are there settings I can tweak the zfs pool with? I'm a huge noob and I don't even know how or where I could change those settings.

I appreciate every help I could get.
Thanks.

Edit: Some more graphs. http://goo.gl/Lg2iy
And I'm using the FreeNAS 8.2-beta4

 

[cyberjock]

Looking at the graphs it appears that when the LAN speeds drop the CPU goes really high(about 100%). I do know that more ram is always better. I do see that your IRQ maxes out at almost 50%(1 whole CPU). I may be mistaken but that seems REALLY high.


Random thoughts you can try...

Do you have AHCI and other performance optimizing settings in the BIOS enabled?

Can you install more than 8GB temporarily and see what happens?

Personally, I'm not an AMD person and I never recommend AMDs, but I'm trying to be objective. It COULD be your CPU, but I really can't argue my opinion with facts. Perhaps someone else with an equivalent CPU could comment on it.

I may be mistaken, but alot of realtek NICs suck(if not all of them). It is highly recommended that FreeNAS users use an Intel NIC. It is possible that the realtek NIC is using alot of IRQs because it sucks and that is also translating to a performance hit. Does your network card support offloading? Pretty much all Intel NICs are compatible with FreeNAS and they can be bought for less than $30 with shipping.

 

[detyrious]

Thanks for your reply. I looked over the BIOS settings and AHCI is activated for the onboard SATA controller.
I also ordered 16GB RAM modules (the last batch was faulty, damn you corsair) and an Intel Server NIC (PCIe). That should take a little pressure off the CPU.

Still, I don't get it why a three disk raid-z works perfectly well with constant data transfer (~70MB/s) while a four disk raid-z has so many problems.

I appreciate any further tips or zfs tuning settings I could try (for 8GB and 16GB RAM).

 

[cyberjock]

Have you tried replacing the 4th drive with one of the other 3 and doing a RAIDZ to see what happens? It could be that the 4th disk is faulty. It could also be a bad cable that connects to the 4th drive.

Typically you shouldn't need to tune ZFS. The default parameters work for most people. If you have a very specific use for it(databases for example) then you could tune it to optimize the performance, but generally it is not necessary.

 

[detyrious]

Hi there,

I tried the four disks in every configuration possible, so neither the cables nor the disks should be faulty - it's the same with every configuration: three disk raid-z works fine, four disk raid-z stutters like hell.

Here a new graph from today (new Intel server NIC): http://goo.gl/P7QDw
To the left of the gap is a three disk raid-z and to the right the four disk raid-z. It's better than before for sure, but not perfect.
I think either the CPU or the onboard SATA controller are limiting the bandwith.

Sooo some suggestions for future hardware upgrades? 16GB RAM are coming in, but what about CPU and mainboard? I'd need an ITX board, so the market is not that huge.
Any suggestions? ITX board with so. 1155 and a Core i5? Would be pretty costly (>200euros) and I don't know about temperatures in such a small ITX case.

Some help? :)

 

[William Grzybowski]

I'm not surprised 3 disk raidz is faster.

This is the optimal configuration for raidz... 2^n+1 = 3, 5, 9, 17...
There is a considerable penalty otherwise.

 

[detyrious]

So, I tried some things in the last few days.
The following tuneables improve the performance greatly (much lesser stalls and stutters):
vfs.zfs.txg.write_limit_override = 492830720
vfs.zfs.txg.timeout = 5

It's not perfect, I have to admit. But with those settings I get a steady write speed of around 55MB/s.

 

[ProtoSD]

It seems like you just completely disregarded what William posted above. There is a whole thread as well as a FAQ question about ZFS performance related to the number of disks and whether you're using raidz or raidz2.

Raidz works better with an odd number of disks, raidz2 works better with an even number of disks.

 

[andru]

Thank you for sharing the graphs.
I was wondering if i need 8GB or 4 GB will be enogh, and I see that it pretty much uses 6.
I dont use any raid though.