Freenas very slow reads problem ?

[Tywin]

I see there is a standard , but most people don't follow it anyway. I took a look I don't see where it says "mb/s" (I wrote early) stays for megabits instead of megabytes ?!
According to what you send me should be MiB for megabits ,but caps or lower case it doesn't say matters ?!
I seen it mostly like Mbps for megabits most officially used, but anyways it's off topic.
Any info is welcome.

You have misparsed the link. Case absolutely matters in units. "mb/s" isn't listed because it is not particularly useful; it would mean "millibits per second". Even if you ignored the part where bits are atomic, use cases for such a unit would be extremely rare.

1 MB = 1 megabyte = 1e6 B
1 MiB = 1 mebibyte = 1024^2 B
1 Mb = 1 megabit = 1e6 bit = 125e3 B = 125 KB

You can then add "/s" (or, less frequently, "ps") to any of those to mean "per second".

 

[Ericloewe]

I'd add that "/s" or "s^-1" is proper scientific notation, while "ps" is something only seen from people who are talking about data rates.

 

[Black Ninja]

What's the read speed of your stripped pool.

Little over 300 mbytes/sec is the Read speed.

 

[Black Ninja]

You have misparsed the link. Case absolutely matters in units. "mb/s" isn't listed because it is not particularly useful; it would mean "millibits per second". Even if you ignored the part where bits are atomic, use cases for such a unit would be extremely rare.

1 MB = 1 megabyte = 1e6 B
1 MiB = 1 mebibyte = 1024^2 B
1 Mb = 1 megabit = 1e6 bit = 125e3 B = 125 KB

You can then add "/s" (or, less frequently, "ps") to any of those to mean "per second".

Thanks for clearing that up. I will try to refer to it properly.

 

[solarisguy]

@Black Ninja, it is easier to remember, when you consider that 9 mm caliber (9 millimeter) is a rather small weapon, and 2000 MW (Mega Watts) is a large power plant

:)

 

[Black Ninja]

What's the read speed of your stripped pool.

Stripped pool can read/write 300MB/s at least but over 1GB I can only max write speed around 110MB/s but read is barely pushing 92MB/s, I don't know what is wrong !?

 

[bestboy]

Yes, but this might not be the fault of the pool. If the pool can write at 115 MB/s with the dd tests discussed above, it should also be able to write 115 MB/s from the network. So these problems are likely to be network related. And here we have 100 possible causes that may prevent you from better throughput:

#1: using WIFI
#2: using a Realtek NIC
...
#100: bad cabling

You can try to benchmark your network with iperf.
And if you are using a windows client, you can take a look at TCP Optimizer.
FreeNAS usually does not need to be tweaked unless you want to implement a use case that is untypical for a NAS (like WAN access).

 

[Black Ninja]

This is what i use:
CHASSIS: Supermicro CSE-836TQ-R800B ( Direct attached backplane)
MB: Supermicro X9SCM-F
CPU: Xeon 1230 v2
RAM: 32 GB ECC Kingston ( from approved list)
HBA: 2xLSI9211 flashed in IT mode (with p16 Firmware)
HDD: 7x2TB HGST and 1 Intel SSD 530 120GB
FreeNAS-9.3-STABLE

It has dual Intel server grade nics, so no Realtek. As fas as the WIFI - If I use Wifi I won't even bother to ask help. I have CAT6 solid 23AWG running 45FT. I wonder what other people are getting with the same MB ?!

 

[Black Ninja]

By the way pool can Write 110MB/s very easy almost in any pool combination, the problem is when you read from poll struggle to reach 97MB/s . I just mention it because you wrote "write" , but I said above Write is perfectly fine , Read from pool is the problem ?!

 

[bestboy]

erm.. yes. reads, writes. Doesn't really matter.

I was aware of your server specs from post #1. They are more than solid, which is why my concern about Realtek NICs was directed at your test client. We often have users complain about their pool being slow, when in reality their client cannot keep up. Often the problem is an onboard NIC. Sometimes it's even that the hard drive cannot read/write constantly at 120 MB/s.
My point is: Since your server seems fine, I'd take a look at the client and/or the network.

 

[Black Ninja]

I am not sure if it matters, because I don't know what is causing it , I just wanted to be clarify if somebody has the same issue.

The client machine is 4930K,32GB Quad channel DDR3, 3 SSD's no mechanical drives, and it's intel nic 82579V plus additional Dual 1000PRO/PT.

Also I did test from another machine i3, 4GBRAM with in Intel nic 217V

By the way I was able to saturate 2x1Gbit links writing to the nas simultaneous with 110 MB/s , but I cant's saturate a single Gigabit link when READ from NAS :-(

 

[bestboy]

That client also seems pretty solid. It should have no trouble maxing out GbE.

On the server you can check for the typical issues: retransmissions, duplicate acks, out-of-order packets, reduced congestion window, etc

Code:

netstat -s
sysctl net.inet.tcp.hostcache.list

If all that does not come up with a clue, you can try to increase the TCP send buffers on the server by adding the sysctl

Code:

net.inet.tcp.sendspace

with a value of 65536 (default is 32768).

Other than that you can take a look at a transfer with Wireshark and inspect the TCP windows.

BTW: How do you test the throughput?

 

[Black Ninja]

I test with copying files over the network around 68GB total in 3-4GB files.

 

[bestboy]

I copy my files with

Code:

[ ] NFS
[ ] CIFS
[ ] AFP
[ ] FTP
[ ] RSYNC
[ ] SCP
[ ] iSCSI
[ ] something else: ______

:D

 

[Black Ninja]

I am sorry CIFS with smb2.0

 

[Black Ninja]

I tried so many things on my end , this could be bug in 9.3 . Perhaps the people that don't have this problem were still using older Freenas version. 2 Machines totally different generations one is X9 other one is X7 and doing the same thing ! I will try older nas version if nobody has any other ideas.