Network file transfer is very slow

[GrandBleu]

Hello,
I built FreeNAS box with the help from here and it's been working okay until recently. File copy transfer rate used to be around 40-80MB, but it's dropped down to 2-4MB/s. I tried to reset NIC config but didn't help. I tried iperf -s command but nothing displays after below

1. ------------------------------------------------------------
2. Server listening on TCP port 5001
3. TCP window size: 85.3 KByte (default)
4. ------------------------------------------------------------

It seems to hang after that. Ping command works even though each ping takes about 9ms. Can anyone tell/give me what to do to diagnose? It is connected to gigabit switch and router. My FreeNAS box info is below and I'm using CIFS to share to Windows 7.

FreeNAS-9.2.1.9
CPU: Intel i3-433040
Motherboard: X10SLM+-F
Memory: Crucial 2x8gb CT2kit102472BD160B
PSU: Seasonic G series SSR-550RM
HDD: 6 x WD 3TB RAIDZ2)
CASE: Fractal Design R4

TIA.

 

[SweetAndLow]

After iperf - s you need to test using a client that runs iperf.

Also look at local disk reads and writes, what protocol and client could be bottleneck.

 

[GrandBleu]

Hi SweetAndLow,
Can I try the client from the FreeNAS server on the same machine?
How can I find the local disk reads and writes?
How can I look at what protocol and client could be bottleneck?

 

[SweetAndLow]

Iperf needs to be done using another computer in the between just search Google for how to do that.

Local disk speeds can be tested by using dd command. Make sure the file you write is larger than the amount of memory you have and turn compression off.

Protocol should be easy, what sharing option are you using? That is your protocol.

 

[GrandBleu]

C:\>iperf -c <ip-address>
------------------------------------------------------------
Client connecting to <ip-address>, TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 3] local <ip-address> port 9162 connected with <ip-address> port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 41.5 MBytes 34.6 Mbits/sec

This seems to be way low afaik. What can I do to improve this? I noticed that in Supermicro configuration page which I think is IPMI GUI 'Network Link Status' speed is 100M instead of 1000M. How can I change it to gigabit?


Also, I ran dd if=/dev/zero of=testfile bs=1024 count=50000 and got below message
dd: testfile: Read-only file system

I have 16G of memory and CIFS sharing. Could you tell me what's wrong with this dd command? Or example dd command to test disk speed I should use?

 

[SweetAndLow]

You are getting better speeds than 100Mb but you should check your networking. Make sure ask your switches are 1Gb and your cables are cat5e or better. My first guess is you are using WiFi. Using WiFi and expecting performance isn't something that happens. WiFi doesn't offer any kind of performance.

Example dd command: dd if=/dev/zero of=/mnt/pool/dataset bs=256M count=64

 

[GrandBleu]

You are getting better speeds than 100Mb but you should check your networking. Make sure ask your switches are 1Gb and your cables are cat5e or better. My first guess is you are using WiFi. Using WiFi and expecting performance isn't something that happens. WiFi doesn't offer any kind of performance.

Example dd command: dd if=/dev/zero of=/mnt/pool/dataset bs=256M count=64

I'm not quite following. I got 34.6 Mb/s and don't know how it's better speed than 100Mb. Could you explain this?

My FreeNAS box has only wired connection and I don't think it has WiFi interface anyway since it's using on-board network interface. What am I missing here?

Thanks for the dd command! I'll run it when I get access to my FreeNAS later and let you know. I'm not sure why I couldn't create a file under /dev as root. Am I having some permission issue on my FreeNAS?

I have cat5e cables mostly in my house but I'll double check. I know my new switch is gigabit one. How can I ask(or make sure) it is 1Gb?

 

[DifferentStrokes]

Mb = megabits
MB = megabytes

 

[Ericloewe]

Mb = megabits
MB = megabytes

iPerf clearly states MBits/s, though.

I'm not quite following. I got 34.6 Mb/s and don't know how it's better speed than 100Mb. Could you explain this?

My FreeNAS box has only wired connection and I don't think it has WiFi interface anyway since it's using on-board network interface. What am I missing here?

Thanks for the dd command! I'll run it when I get access to my FreeNAS later and let you know. I'm not sure why I couldn't create a file under /dev as root. Am I having some permission issue on my FreeNAS?

I have cat5e cables mostly in my house but I'll double check. I know my new switch is gigabit one. How can I ask(or make sure) it is 1Gb?

It's either Gigabit or it isn't. If you say it is, that should be okay.

In order of likelihood:

• Bad cables
• Absolutely atrocious network adapter on the client end
• Defective switch (possibly just one of the ports)

 

[DifferentStrokes]

iPerf clearly states MBits/s, though.

Yeah. He had asked how 34.6 MB/s is greater than 100Mb/s.

 

[GrandBleu]

iPerf clearly states MBits/s, though.


It's either Gigabit or it isn't. If you say it is, that should be okay.

In order of likelihood:

• Bad cables
• Absolutely atrocious network adapter on the client end
• Defective switch (possibly just one of the ports)

Thanks Ericloewe! I'll check those three things when I get home. I remember you helped me when I built FreeNAS box.

 

[GrandBleu]

Yeah. He had asked how 34.6 MB/s is greater than 100Mb/s.

Here's what I asked.
I got 34.6 Mb/s and don't know how it's better speed than 100Mb.

I've seen people usually get hundreds of Mb on that output, so I wanted to ask about it to figure out what's going on. Let me know if that's normal speed range.

 

[Ericloewe]

Assuming everything is well, expect very close to 1000Mbit/s, 950Mbit/s+.

Scale those numbers accordingly if you're using 10/100.

This particular situation reeks of insanely bad cabling. Even Realteks push beyond 500Mbit/s without any trouble.

 

[DifferentStrokes]

Here's what I asked.
I got 34.6 Mb/s and don't know how it's better speed than 100Mb.

I've seen people usually get hundreds of Mb on that output, so I wanted to ask about it to figure out what's going on. Let me know if that's normal speed range.

My apologies. I could have swear I seen "B" earlier today. Maybe I need more caffeine. :)

 

[GrandBleu]

I replaced cat5e cable between FreeNAS and gigabit switch with new cat5e cable. Also, fully power cycled computers, switch, router and cable modem. Iperf test shows [ 4] 0.0-10.0 sec 60.0 MBytes 50.3 Mbits/sec a bit better than before but still not even close to where it was before.

I tried to copy file via Windows Explorer and speed was around 5MB which is better than 2-3 MB before.

I ran dd command: dd if=/dev/zero of=/mnt/pool/dataset bs=256M count=64 and this displays

dd: /mnt/pool/dataset: No such file or directory

There's no /mnt/pool directory on my FreeNAS. Is it supposed to be there?

I'm stuck and I just want to be able to copy files around at the speed of better than 50MB which is what I used to have.
Anything I can try... please let me know.

 

[pirateghost]

I replaced cat5e cable between FreeNAS and gigabit switch with new cat5e cable. Also, fully power cycled computers, switch, router and cable modem. Iperf test shows [ 4] 0.0-10.0 sec 60.0 MBytes 50.3 Mbits/sec a bit better than before but still not even close to where it was before.

I tried to copy file via Windows Explorer and speed was around 5MB which is better than 2-3 MB before.

I ran dd command: dd if=/dev/zero of=/mnt/pool/dataset bs=256M count=64 and this displays

dd: /mnt/pool/dataset: No such file or directory

There's no /mnt/pool directory on my FreeNAS. Is it supposed to be there?

I'm stuck and I just want to be able to copy files around at the speed of better than 50MB which is what I used to have.
Anything I can try... please let me know.

You are supposed to replace /mnt/pool with the name of YOUR pool. You probably shouldn't be running commands like this on your system. It sounds like you are about to blow something up.

 

[Pheran]

Let's verify the NIC speeds. From FreeNAS, post the output of:

ifconfig

From Windows, post the output of:

ipconfig

wmic NIC where NetEnabled=true get Name,Speed

Something funny is going on because 9ms pings are not normal for a LAN. I can do better than that over wireless. In fact post your ping output too.

 

[zoomzoom]

My apologies. I could have swear I seen "B" earlier today. Maybe I need more caffeine. :)

You did see a B... "41.5 MBytes 34.6 Mbits/sec"

Here's what I asked.
I got 34.6 Mb/s and don't know how it's better speed than 100Mb.

I've seen people usually get hundreds of Mb on that output, so I wanted to ask about it to figure out what's going on. Let me know if that's normal speed range.

It's extremely important with the issue you're having to double check you're using the correct terminology of Mb (MegaBit) vs MB (MegaByte).

• 1 Mb = 0.125 MB
• 10 Mb = 1.25 MB
• 100 Mb = 12.5 MB
• 1000 Mb (1 Gb) = 125 MB

 

[GrandBleu]

You are supposed to replace /mnt/pool with the name of YOUR pool. You probably shouldn't be running commands like this on your system. It sounds like you are about to blow something up.

Thanks for correcting me on this! I found below from FreeNAS doc.
The FreeNAS® graphical interface uses the term volume to refer to a ZFS pool.

I replaced '/mnt/pool' with '/mnt/<my pool/volume name>' in the command and ran it. It shows

64+0 records in
64+0 records out
17179869184 bytes transferred in 4.462328 secs ( 3849979058 bytes/sec )

This result seems to be showing gigabit transfer locally.

 

[GrandBleu]

Let's verify the NIC speeds. From FreeNAS, post the output of:

ifconfig

From Windows, post the output of:

ipconfig

wmic NIC where NetEnabled=true get Name,Speed

Something funny is going on because 9ms pings are not normal for a LAN. I can do better than that over wireless. In fact post your ping output too.

From FreeNAS ifconfig:
ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
ether 0c:c4:7a:0c:d7:26
inet 192.168.1.103 netmask 0xffffff00 broadcast 192.168.1.255
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO>
ether 0c:c4:7a:0c:d7:27
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
status: no carrier
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

From Windows's ipconfig:
Windows IP Configuration


Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . :
Link-local IPv6 Address . . . . . : fe80::19ac:783c:96c0:f2b3%10
IPv4 Address. . . . . . . . . . . : 192.168.1.100
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.1.1

Tunnel adapter isatap.{F0265E41-81C2-4E23-A82A-E581E1791101}:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :

Tunnel adapter Teredo Tunneling Pseudo-Interface:

Media State . . . . . . . . . . . : Media disconnected
Connection-specific DNS Suffix . :

From the wmic command:
Name Speed
Realtek PCIe GBE Family Controller 1000000000