Remote Connections Limited to 10 mbps

[kdragon75]

Try using UDP, 2 streams, and a 64KByte window size. The run the same test with TCP.

EDIT: run this from the FreeNAS server, not your desktop.

Er... Are you testing this via NAT reflection? Put another way, are you running iperf on your desktop from your LAN to the your WAN IP(dynamic DNS name)? if this is essentially a loopback test that could be impacted by ISP QOS of some sort or even just a crappy NAT implementation in your TPLink.

 

[TheSunKing]

EDIT: run this from the FreeNAS server, not your desktop.

Err, wait - now I'm confused. What's the point of running the iperf client on the FreeNAS server if the iperf server is also running on the FreeNAS server? Wouldn't it just be short-circuiting and not really testing the network?
I'm running the tests now, I'm just trying to understand the logic behind it.

For clarification, the server is in Ohio and the client PC (with the slow transfer speed from the server that I'm trying to resolve) is in West Virginia. So they're on completely different networks.

 

[TheSunKing]

Running server and client on FreeNAS box:


UDP

iperf -c redacted.duckdns.org -u -P 2 -i 1 -p xxxxx-w 64.0K -f M -b 2.5M -t 60 -T 1

------------------------------------------------------------
Client connecting to redacted.duckdns.org, UDP port xxxxx
Sending 1470 byte datagrams, IPG target: 4486.08 us (kalman adjust)
UDP buffer size: 0.06 MByte
------------------------------------------------------------
[ 3] local 192.168.0.xxx port 26417 connected with 74.215.xxx.xx port xxxxx
[ 4] local 192.168.0.xxx port 15733 connected with 74.215.xxx.xx port xxxxx
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 0.31 MBytes 0.31 MBytes/sec
[ 4] 0.0- 1.0 sec 0.31 MBytes 0.31 MBytes/sec
[SUM] 0.0- 1.0 sec 0.63 MBytes 0.63 MBytes/sec
[ 3] 1.0- 2.0 sec 0.31 MBytes 0.31 MBytes/sec
[ 4] 1.0- 2.0 sec 0.31 MBytes 0.31 MBytes/sec
[SUM] 1.0- 2.0 sec 0.63 MBytes 0.63 MBytes/sec
...
[ 4] 0.0-59.9 sec 18.7 MBytes 0.31 MBytes/sec
[ 4] Sent 13359 datagrams
[ 3] 0.0-60.0 sec 18.8 MBytes 0.31 MBytes/sec
[ 3] Sent 13376 datagrams
[SUM] 0.0-60.0 sec 37.5 MBytes 0.62 MBytes/sec
[SUM] Sent 26735 datagrams
[ 4] WARNING: did not receive ack of last datagram after 10 tries.
[ 3] WARNING: did not receive ack of last datagram after 10 tries.



TCP

iperf -c redacted.duckdns.org -P 2 -i 1 -p xxxxx -w 64.0K -f M -t 60

------------------------------------------------------------
Client connecting to redacted.duckdns.org, TCP port xxxxx
TCP window size: 0.06 MByte (WARNING: requested 0.06 MByte)
------------------------------------------------------------
[ 3] local 192.168.0.xxx port 46814 connected with 74.215.xxx.xx port xxxxx
[ 4] local 192.168.0.xxx port 14300 connected with 74.215.xxx.xx port xxxxx
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 13.6 MBytes 13.6 MBytes/sec
[ 4] 0.0- 1.0 sec 13.6 MBytes 13.6 MBytes/sec
[SUM] 0.0- 1.0 sec 27.2 MBytes 27.2 MBytes/sec
[ 3] 1.0- 2.0 sec 13.5 MBytes 13.5 MBytes/sec
[ 4] 1.0- 2.0 sec 13.5 MBytes 13.5 MBytes/sec
...
[ 4] 0.0-60.0 sec 806 MBytes 13.4 MBytes/sec
[ 3] 0.0-60.0 sec 806 MBytes 13.4 MBytes/sec
[SUM] 0.0-60.0 sec 1612 MBytes 26.9 MBytes/sec


Interesting. I expected both to get much faster, given that the traffic shouldn't actually be hitting the internet, but only TCP got faster.

 

[TheSunKing]

Running server on FreeNAS box and client on PC:

UDP = 0.60 MB/s

TCP = 0.72 MB/s

 

[Elliot Dierksen]

Yup, I used iperf/jperf before posting. I actually made a typo in my diagram - ioperf should be iperf. I got only 0.7 MB/s (6 mbps) in the iperf test.

Are hosts experiencing the problem directly connected to the wifi in your local router, or is that a remote network connection across an ISP connection? If it is the former, you could have some kind of wifi interference. If it is the latter, you are only as fast as the slowest link along the path. If it is the latter, how hops does a traceroute show and what are the response times? I would be particularly interested in the slowest hop.

 

[TheSunKing]

Are hosts experiencing the problem directly connected to the wifi in your local router, or is that a remote network connection across an ISP connection?

It's the latter.

17 hops to hit the server.

Traceroute using nmap:

TRACEROUTE (using proto 1/icmp)
HOP RTT ADDRESS
1 14.00 ms 96.120.104.1
2 13.00 ms te-4-2-sr01.redacted.wv.bad.comcast.net (68.87.133.181)
3 14.00 ms xe-4-1-0-0-sur01.newmexiconw.dc.bad.comcast.net (68.85.67.205)
4 14.00 ms 68.87.136.13
5 23.00 ms 69.139.246.185
6 20.00 ms be-33657-cr02.ashburn.va.ibone.comcast.net (68.86.90.57)
7 18.00 ms be-10130-pe04.ashburn.va.ibone.comcast.net (68.86.82.214)
8 19.00 ms 50.248.118.174
9 20.00 ms be3083.ccr41.dca01.atlas.cogentco.com (154.54.30.53)
10 36.00 ms be2891.ccr21.cle04.atlas.cogentco.com (154.54.82.249)
11 38.00 ms be2732.rcr21.cmh02.atlas.cogentco.com (154.54.40.249)
12 39.00 ms be2730.rcr51.cvg02.atlas.cogentco.com (154.54.40.177)
13 39.00 ms 154.24.58.166
14 40.00 ms 38.122.234.102
15 40.00 ms 216.68.14.104
16 42.00 ms mh1-dsl-redacted.fuse.net (redacted)
17 47.00 ms ws1-dsl-redacted.fuse.net redacted)


 

[Elliot Dierksen]

So none of the times are bad, in fact they are pretty good. It is still a fair number of hops. I don't know if there are some options in iperf that would let you increase the window size (aka amount of unacknowledged data before stopping transmission), but that would be worth investigating. There are enough hops that just the propagation delay (time it takes the bits in transit) could make you pause if the TCP window size is exceeded. What do you get on generic internet speed tests at the remote site?

 

[TheSunKing]

What do you get on generic internet speed tests at the remote site?

69 mbps down, 5 mbps up

 

[Elliot Dierksen]

69 mbps down, 5 mbps up

That isn't bad, but I am fairly certain you are getting slowed down by the TCP window size.

https://kb.wisc.edu/uwsysnet/page.php?id=41705

Try increasing the window size to 128K on both sides and see if that changes the results.

 

[kdragon75]

So none of the times are bad, in fact they are pretty good. It is still a fair number of hops. I don't know if there are some options in iperf that would let you increase the window size (aka amount of unacknowledged data before stopping transmission), but that would be worth investigating. There are enough hops that just the propagation delay (time it takes the bits in transit) could make you pause if the TCP window size is exceeded. What do you get on generic internet speed tests at the remote site?

Thats what I was thinking with the UDP test.

 

[Stux]

I recall reading something about iperfs udp tests being broken...

 

[TheSunKing]

Try increasing the window size to 128K on both sides and see if that changes the results.

The server side command wouldn't let me specify a window size. I believe it's adaptive based on the client command.
Using: 128K, 1 stream; 128K, 2 streams; 256K, 1 stream; and 256K, 2 streams on client side => 0.72 MB/s in all cases.

 

[Elliot Dierksen]

The server side command wouldn't let me specify a window size. I believe it's adaptive based on the client command.
Using: 128K, 1 stream; 128K, 2 streams; 256K, 1 stream; and 256K, 2 streams on client side => 0.72 MB/s in all cases.

Are you running the server side at the local end, or the remote end? Whichever side is sending the data stream, that is the side where the TCP window size matters. .72MB is in the vicinity of the 5Mb upload speed at the remote. You could either be hitting that, or there is a slow/congested link between the local and the remote.

 

[TheSunKing]

Are you running the server side at the local end, or the remote end?

Server side was running on the server, so the 'local' end in my original diagram.

 

[Elliot Dierksen]

Server side was running on the server, so the 'local' end in my original diagram.

I can't help but wonder if the server is receiving the transmission from the client, and not the other way around. That would then put the ~= 5MB at the upload speed of the remote.