Slow speed: can't find the problem....

[erick vogeler]

Hi

so here are some details, i am getting under 3 MB/s when according to speed test i should be getting 38 Mb/s. this i have tried on several servers / desktops with same result.

Here is some info:

Build FreeNAS-9.3-STABLE-201602031011
Platform Intel(R) Xeon(R) CPU E5530 @ 2.40GHz
Memory 12251MB
System Time Sun Mar 06 12:33:50 EST 2016
Uptime 12:33PM up 9 days, 15:16, 1 user
Load Average 0.03, 0.02, 0.00

Server:

[root@nas] /mnt/Main/win2# dd if=/dev/zero of=temp.dat bs=1024k count=25k
25600+0 records in
25600+0 records out
26843545600 bytes transferred in 15.208945 secs (1764984063 bytes/sec)


[root@nas] ~# /.speedtest-cli
/.speedtest-cli: Command not found.
[root@nas] ~# ./speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from VolumeDrive (xxx.xxx.xxx.xxx)...
Selecting best server based on latency...
Hosted by PenTeleData (Palmerton, PA) [72.43 km]: 25.726 ms
Testing download speed........................................
Download: 308.66 Mbit/s
Testing upload speed..................................................
Upload: 183.43 Mbit/s


[root@nas] ~# ifconfig
bce0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 00:00:00:00:00:xx
inet xxx.xxx.xxx.xxx netmask 0xffffff00 broadcast xxx.xxx.xxx.xxx
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
bce1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 00:00:00:00:00:xx
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0xb
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Appreciate the help

Thanks

 

[nightshade00013]

First off you don't have half the information you should for us to help. Drives, pool type etc would be a great help.

Second off it looks like you did the DD in a dataset that has compression turned on. Create another dataset with it off and do the DD test there. You are getting over 1GB/s in your test and unless you have some SSD's or multiple vDev's that isn't correct.

 

[Ericloewe]

according to speed test i should be getting 38 Mb/s

Please explain what on Earth you mean.

 

[Bhoot]

Hi

so here are some details, i am getting under 3 MB/s when according to speed test i should be getting 38 Mb/s.
a whole lot of snips
Testing download speed........................................
Download: 308.66 Mbit/s
Testing upload speed..................................................
Upload: 183.43 Mbit/s
another lot of snips

Tada.

 

[Ericloewe]

Hi

so here are some details, i am getting under 3 MB/s when according to speed test i should be getting 38 Mb/s. this i have tried on several servers / desktops with same result.

Here is some info:

Build FreeNAS-9.3-STABLE-201602031011
Platform Intel(R) Xeon(R) CPU E5530 @ 2.40GHz
Memory 12251MB
System Time Sun Mar 06 12:33:50 EST 2016
Uptime 12:33PM up 9 days, 15:16, 1 user
Load Average 0.03, 0.02, 0.00

Server:

[root@nas] /mnt/Main/win2# dd if=/dev/zero of=temp.dat bs=1024k count=25k
25600+0 records in
25600+0 records out
26843545600 bytes transferred in 15.208945 secs (1764984063 bytes/sec)


[root@nas] ~# /.speedtest-cli
/.speedtest-cli: Command not found.
[root@nas] ~# ./speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from VolumeDrive (xxx.xxx.xxx.xxx)...
Selecting best server based on latency...
Hosted by PenTeleData (Palmerton, PA) [72.43 km]: 25.726 ms
Testing download speed........................................
Download: 308.66 Mbit/s
Testing upload speed..................................................
Upload: 183.43 Mbit/s


[root@nas] ~# ifconfig
bce0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 00:00:00:00:00:xx
inet xxx.xxx.xxx.xxx netmask 0xffffff00 broadcast xxx.xxx.xxx.xxx
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
bce1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c01bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
ether 00:00:00:00:00:xx
nd6 options=9<PERFORMNUD,IFDISABLED>
media: Ethernet autoselect
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
nd6 options=9<PERFORMNUD,IFDISABLED>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0xb
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Appreciate the help

Thanks

Did you edit your post? If so, please don't just edit your posts adding information. People aren't notified of edits and they'll go unseen.

[root@nas] ~# /.speedtest-cli
/.speedtest-cli: Command not found.
[root@nas] ~# ./speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from VolumeDrive (xxx.xxx.xxx.xxx)...
Selecting best server based on latency...
Hosted by PenTeleData (Palmerton, PA) [72.43 km]: 25.726 ms
Testing download speed........................................
Download: 308.66 Mbit/s
Testing upload speed..................................................
Upload: 183.43 Mbit/s

Speedtest.net is absolutely useless, in most cases, to determine your LAN's performance. Please use iperf to check this kind of thing.

bce0:

Ugh, Broadcom.

 

[Bhoot]

Did you edit your post? If so, please don't just edit your posts adding information. People aren't notified of edits and they'll go unseen.

That's why i personally like quoting and snipping the posts.

 

[erick vogeler]

First of all, let me add that the NAS was working great at home before sending out to datacenter i tested it with local Lan, got 100 Mb/s + with CIF and Compression on, very similar NFS with that ugly broadcom you all hate. Now let me try to answer all your questions.

First off you don't have half the information you should for us to help. Drives, pool type etc would be a great help.

Second off it looks like you did the DD in a dataset that has compression turned on. Create another dataset with it off and do the DD test there. You are getting over 1GB/s in your test and unless you have some SSD's or multiple vDev's that isn't correct.

--- Drivers are all stock.

--- Pool Type: (maybe this not sure:)

[root@nas] /mnt/Main/test1# zpool status
pool: Extra
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKS UM
Extra ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/6deb783e-d2f4-11e5-8f48-e41f1366b440 ONLINE 0 0 0
gptid/764f5b63-d2f4-11e5-8f48-e41f1366b440 ONLINE 0 0 0

errors: No known data errors

pool: Main
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKS UM
Main ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/47fabeec-d181-11e5-822c-e41f1366b440 ONLINE 0 0 0
gptid/4a2e13e5-d181-11e5-822c-e41f1366b440 ONLINE 0 0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
da4p2 ONLINE 0 0 0
da5p2 ONLINE 0 0 0

errors: No known data errors

---- DF on new dataset without compression:

dd if=/dev/zero of=temp.dat bs=1024k count=25k
25600+0 records in
25600+0 records out
26843545600 bytes transferred in 271.801831 secs (98761460 bytes/sec)

Please explain what on Earth you mean.

(acording to speed test) 308.66 Mbit/s /8 = 38.58 Mb/s roughly but am only getting max 3 Mb/s (Yes i know speedtest is not reliable but you can get numbers very close to reality)

Got it?

Tada.

Tada for you too whatever it means

Did you edit your post? If so, please don't just edit your posts adding information. People aren't notified of edits and they'll go unseen.

Speedtest.net is absolutely useless, in most cases, to determine your LAN's performance. Please use iperf to check this kind of thing.

Ugh, Broadcom.

Yes edited withing seconds, didn't feel the need.

That is correct however more close to reality if you are planing transfers over the Internet, and not over LAN's (over LAN worked great tho)

anyways i'll get back to you with the iperf result

 

[Ericloewe]

You mean the server is at a datacenter? Well, that's a very important detail.

God knows what kind of route your data has to take across the internet. Just because you have a decent connection to the nearest speedtest server doesn't mean you'll have it all the way to your home.

 

[erick vogeler]

You mean the server is at a datacenter? Well, that's a very important detail.

God knows what kind of route your data has to take across the internet. Just because you have a decent connection to the nearest speedtest server doesn't mean you'll have it all the way to your home.

Am not expecting the 100 Mb/s i get home, but 3MB/s is unbearable

 

[erick vogeler]

also the use is not for home, i am getting the same connection from OVH servers and other datacenters i use

 

[Bhoot]

Am not expecting the 100 Mb/s i get home, but 3MB/s is unbearable

The only way to ensure that is to get a dedicated line from the server to your location access..
Just FYI 3MB/s is roughly 25 Mb/s.

I haven't really transferred stuff across the internet but I have been able to stream 1080p files through plex remotely without a plex pass with a 60Mb/s connection speed.

 

[erick vogeler]

Well i need a little help with the iperf,

i did iperf -s on the nas seems ok

on the other server (centos 7 no firewall ovh datacenter) i do iperf3 -c "ipadress"

and get iperf3: error - unable to connect to server: Connection refused

does freenas comes with firewall?

 

[erick vogeler]

The only way to ensure that is to get a dedicated line from the server to your location access..
Just FYI 3MB/s is roughly 25 Mb/s.

I haven't really transferred stuff across the internet but I have been able to stream 1080p files through plex remotely without a plex pass with a 60Mb/s connection speed.

Yea i have to transfer 30gb once a day, thats only backup for site, then VM's images, isos, more backups etc etc etc

 

[Ericloewe]

does freenas comes with firewall?

No, not at all.

Are you using compatible versions of iperf? FreeNAS doesn't use the latest, which is incompatible with FreeNAS' version.

 

[erick vogeler]

No, not at all.

Are you using compatible versions of iperf? FreeNAS doesn't use the latest, which is incompatible with FreeNAS' version.

oh yea i was using iperf3, you have any link where to download the correct version please?

 

[Ericloewe]

https://iperf.fr/iperf-download.php

You'll want a version 2.something

 

[erick vogeler]

[root@ns238538 ~]# iperf -c xxxxxx
------------------------------------------------------------
Client connecting to xxxxxx, TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 3] local 1zzzzzzxxxxx port 28519 connected with xxxx port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.1 sec 106 MBytes 88.6 Mbits/sec

This is from server to server (not my home)