Weird transfer problem, need help debugging.

[SnorreSelmer]

Hello all.

Today I discovered a weird issue I find very puzzling.

I lent my server to a friend so he can dump his Ubuntu server and do a full install of the latest version. First we tried transferring using CIFS, but that maxed out at ~300Mbit/sec. Then we tried FTP and got an instant reward of ~900Mbit/sec... For about 5-30 sec at a time! We did an iostat to see if we were maxing the disks themselves, but we weren't (they were operating at ~10MB/sec, at least 1/6th of what they can do). It seems the network isn't the issue either, because the time between drop-outs varies a lot. While we're averaging ~500Mbit/sec using FTP, we would like the number to be closer to 900Mbit/sec if possible.

We did notice something when we ran iostat though: different disks stopped writing at different times, so it's as if there's some buffer that fills up, but with the power of the server, I can't think of anywhere that should become a bottleneck. The server uses around 25% CPU capacity (probably the ftpd process that accounts for most of that), the RAM is at ~80% utilization, the drives themselves have loads of transfer capacity available.

Can anyone provide some tips on what we can do to find the bottleneck?

 

[ProtoSD]

There's some ZFS tuning that would probably help. I don't have time to look up all the correct variables, but probably tvfs.zfs.txg.timeout=5 and vfs.zfs.txg.write_limit_override=1073741824 will help with the writing. Those are just off the top of my head, there are others, but adding those to Sysctls or Tunables under System in the GUI and rebooting will probably help. There are some CIFS settings also. I've post 2 or 3 other threads about these settings. Search for slow CIFS, abysmal.... Probably the whole group of settings I posted in those threads will help you.

 

[SnorreSelmer]

I found this thread you posted in (http://forums.freenas.org/showthread.php?5338-Browsing-directories-slow&p=26269&viewfull=1#post26269) and implemented the tweaks. It looked like it helped at first but I'm still getting dropouts where the transfer-speed drops to 0B/sec.

When I get the server back I'll try to do some internal and external reading and writing to see if I can narrow the problem down to storage or network.