Doubleslash
Cadet
- Joined
- Mar 12, 2013
- Messages
- 5
Hi community,
I am running the famous ProLiant N36L Microserver with 8GB RAM and 4x2TB HDDs (2xSamsung F4 and 2xWD20EARX). I have a large volume created from a ZFS RAID0 stripe over 2 ZFS RAID1 mirrors, no RAIDZ. I have not create a dataset or something
Using xdd or dd I can get around 200 MB/s read and write from empty ARC caches (after fresh boot) from a 5G file using sequential access patterns. Also monitoring the network performance with iperf between my i7 iMac and the FreeNAS box is around 116 MB/s in each direction, using a 1GbE network connection and a switch. So far so good from the hardware side.
The problem arises when I am attempting use any kind of networking server to read and write large files sequentially. No matter if it is FTP, SMB or AFP... I always top out at around 60 MB/s read. Writes are very fast, with SMB being the fastest at constantly 100 MB/s write testing from a Windows 8 machine using an SSD. The iMac also has an SSD and achieves 100 MB/s write as well with some bumps inbetween.
Anyhow: using both Windows 8 (SMB) and Mac OS X 10.8 or generic protocols (FTP)... reads are limited to 60 MB/s. Any idea why that is? What could it slow down from raw 200 MB/s to 60 MB/s over the network?
I use FreeNAS 8.1.3-RC1 x64 and have the following sysctls set:
Variable Value Comment Enabled Available actions
kern.ipc.maxsockbuf 16777216
kern.ipc.nmbclusters 32768
kern.ipc.shmall 32768
kern.ipc.shmmax 67108864
kern.ipc.somaxconn 8192
net.inet.tcp.delayed_ack 0
net.inet.tcp.hostcache.expire 1
net.inet.tcp.inflight.debug 0
net.inet.tcp.inflight.enable 0
net.inet.tcp.path_mtu_discovery 1
net.inet.tcp.recvbuf_auto 1
net.inet.tcp.recvbuf_inc 524288
net.inet.tcp.recvbuf_max 16777216
net.inet.tcp.recvspace 262144
net.inet.tcp.rfc1323 1
net.inet.tcp.sack.enable 1
net.inet.tcp.sendbuf_auto 1
net.inet.tcp.sendbuf_inc 8192
net.inet.tcp.sendbuf_max 16777216
net.inet.tcp.sendspace 262144
net.inet.udp.maxdgram 65535
net.inet.udp.recvspace 65536
net.local.stream.recvspace 65536
net.local.stream.sendspace 65536
kern.maxfiles 65536
kern.maxfilesperproc 32768
kern.maxvnodes 400000
net.inet.icmp.icmplim 300
net.inet.icmp.icmplim_output 1
net.inet.tcp.mssdflt 1452
vfs.zfs.prefetch_disable 1
Any ideas?
I am running the famous ProLiant N36L Microserver with 8GB RAM and 4x2TB HDDs (2xSamsung F4 and 2xWD20EARX). I have a large volume created from a ZFS RAID0 stripe over 2 ZFS RAID1 mirrors, no RAIDZ. I have not create a dataset or something
Using xdd or dd I can get around 200 MB/s read and write from empty ARC caches (after fresh boot) from a 5G file using sequential access patterns. Also monitoring the network performance with iperf between my i7 iMac and the FreeNAS box is around 116 MB/s in each direction, using a 1GbE network connection and a switch. So far so good from the hardware side.
The problem arises when I am attempting use any kind of networking server to read and write large files sequentially. No matter if it is FTP, SMB or AFP... I always top out at around 60 MB/s read. Writes are very fast, with SMB being the fastest at constantly 100 MB/s write testing from a Windows 8 machine using an SSD. The iMac also has an SSD and achieves 100 MB/s write as well with some bumps inbetween.
Anyhow: using both Windows 8 (SMB) and Mac OS X 10.8 or generic protocols (FTP)... reads are limited to 60 MB/s. Any idea why that is? What could it slow down from raw 200 MB/s to 60 MB/s over the network?
I use FreeNAS 8.1.3-RC1 x64 and have the following sysctls set:
Variable Value Comment Enabled Available actions
kern.ipc.maxsockbuf 16777216
kern.ipc.nmbclusters 32768
kern.ipc.shmall 32768
kern.ipc.shmmax 67108864
kern.ipc.somaxconn 8192
net.inet.tcp.delayed_ack 0
net.inet.tcp.hostcache.expire 1
net.inet.tcp.inflight.debug 0
net.inet.tcp.inflight.enable 0
net.inet.tcp.path_mtu_discovery 1
net.inet.tcp.recvbuf_auto 1
net.inet.tcp.recvbuf_inc 524288
net.inet.tcp.recvbuf_max 16777216
net.inet.tcp.recvspace 262144
net.inet.tcp.rfc1323 1
net.inet.tcp.sack.enable 1
net.inet.tcp.sendbuf_auto 1
net.inet.tcp.sendbuf_inc 8192
net.inet.tcp.sendbuf_max 16777216
net.inet.tcp.sendspace 262144
net.inet.udp.maxdgram 65535
net.inet.udp.recvspace 65536
net.local.stream.recvspace 65536
net.local.stream.sendspace 65536
kern.maxfiles 65536
kern.maxfilesperproc 32768
kern.maxvnodes 400000
net.inet.icmp.icmplim 300
net.inet.icmp.icmplim_output 1
net.inet.tcp.mssdflt 1452
vfs.zfs.prefetch_disable 1
Any ideas?