SOLVED CIFS. Slow Read Speed. Perfect Write Speed

[atlbuckeye]

Hey Guys, FreeNAS noob here. Just finished setting up my freeNAS on Friday. Got 6 drives in RADIZ2 format and I'm having trouble getting acceptable read speeds from my NAS drive. I have mounted the drive and transferred around 2TB to the NAS at gigabit speeds (~110 MB/s), but when I went to test out the read speed by copying a 2GB file form the NAS back to my desktop, I get between 500KB to 2 MB! What the heck?! I don't know where to look for the problem, or where to find a solution. I spent a good amount of time trying to find someone else with this issue, and a solution via google, but to no avail. Specs are below. Help!


Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz
ASRock Micro ATX DDR3 1333 LGA 1150 Motherboards H97M PRO4
5x HITACHI Deskstar 2TB 7200RPM 32MB Cache
1x WD Green 4 TB
8GB Corsair Vengence RAM

Edit: iperf speeds

FreeNAS as server
------------------------------------------------------------
Client connecting to 192.168.1.121, TCP port 5001
TCP window size: 0.03 MByte (default)
------------------------------------------------------------
[ 3] local 192.168.1.142 port 64615 connected with 192.168.1.121 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 2.0 sec 2.75 MBytes 11.5 Mbits/sec
[ 3] 2.0- 4.0 sec 8.50 MBytes 35.7 Mbits/sec
[ 3] 4.0- 6.0 sec 7.00 MBytes 29.4 Mbits/sec
[ 3] 6.0- 8.0 sec 4.88 MBytes 20.4 Mbits/sec
[ 3] 8.0-10.0 sec 3.75 MBytes 15.7 Mbits/sec
[ 3] 0.0-10.0 sec 27.1 MBytes 22.7 Mbits/sec


and PC as Server
---------------------------------------------------
Client connecting to 192.168.1.142, TCP port 5001
TCP window size: 8.00 KByte (default)
---------------------------------------------------
[180] local 192.168.1.121 port 51339 connected with
[ ID] Interval Transfer Bandwidth
[180] 0.0- 2.0 sec 98.1 MBytes 411 Mbits/sec
[180] 2.0- 4.0 sec 97.2 MBytes 408 Mbits/sec
[180] 4.0- 6.0 sec 99.6 MBytes 418 Mbits/sec
[180] 6.0- 8.0 sec 97.5 MBytes 409 Mbits/sec
[180] 8.0-10.0 sec 97.1 MBytes 407 Mbits/sec
[180] 0.0-10.0 sec 489 MBytes 411 Mbits/sec

edit: I am setup in RAIDZ2.

 

[Mirfster]

There is a thread in these forums by cyberjock regarding slow CIFS reads and some parameters you can tune/set. On my cell right now, so not going to search for it and provide the link. Will leave that to you. :)

Also 8GB of RAM is the "minimum recommeded" amount for just running Freenas. Never a bad idea to throw more RAM in there.

 

[atlbuckeye]

There is a thread in these forums by cyberjock regarding slow CIFS reads and some parameters you can tune/set. On my cell right now, so not going to search for it and provide the link. Will leave that to you. :)

Also 8GB of RAM is the "minimum recommeded" amount for just running Freenas. Never a bad idea to throw more RAM in there.

I saw the thread. I interpreted his thread as a "Improve CIFS directory browsing", and not a a read speed improvement. That being said, I would think there is a bigger problem here. If I was at 50-60MB/s, and looking to improve on that, I would be looking at what I can do to tweak. But for now, sitting at 1-3 MB/s, I feel like there are larger fish to fry.

8 GB is all I had laying around when I went to build. I don't plan to keep it at 8 GB forever, but for my array size, I would think that this is sufficient to get a decent amount of performance.

Also, I have since tested a few internal transfers. Copying internally, from NAS to NAS is also at 120MB/s. Copying from NAS to USB 2.0 thumb drive is at 5-6 MB/s

 

[HoneyBadger]

Your iperf results are pointing to a networking issue. The chipset on that board is the Intel i218V which I think is fairly new to FreeBSD support, as in "I think it only showed up as of FreeBSD 10."

Right now, you really don't want to summon Cyberjock into this thread, as you're using very much "consumer-grade" gear. Let's just say he's "rather blunt about his opinions" on that kind of thing.

If I can hazard a guess, this is an old gaming rig you repurposed for FreeNAS, but unfortunately FreeNAS and gaming gear do not play nice at all. Your i5/H97 combo has no support for ECC memory and as noted above a fairly "new-to-BSD" network card. (But at least it's not a Realtek.) If you're only storing data you can afford to lose (downloaded, recorded, or ripped media, your experimental home VMs) then that's fine, but if you're putting vacation or family photos, documents, business records, you're going to have to buy gear suited for the job. Thankfully, it's cheaper than you think to get on the ECC train.

The one Green drive in your RAIDZ2 might also be throwing your overall vdev performance for a bit of a loop. Did you read the thread about using WDIDLE to adjust/disable the spin-down timer on it?

 

[atlbuckeye]

Your iperf results are pointing to a networking issue. The chipset on that board is the Intel i218V which I think is fairly new to FreeBSD support, as in "I think it only showed up as of FreeBSD 10."

Right now, you really don't want to summon Cyberjock into this thread, as you're using very much "consumer-grade" gear. Let's just say he's "rather blunt about his opinions" on that kind of thing.

If I can hazard a guess, this is an old gaming rig you repurposed for FreeNAS, but unfortunately FreeNAS and gaming gear do not play nice at all. Your i5/H97 combo has no support for ECC memory and as noted above a fairly "new-to-BSD" network card. (But at least it's not a Realtek.) If you're only storing data you can afford to lose (downloaded, recorded, or ripped media, your experimental home VMs) then that's fine, but if you're putting vacation or family photos, documents, business records, you're going to have to buy gear suited for the job. Thankfully, it's cheaper than you think to get on the ECC train.

The one Green drive in your RAIDZ2 might also be throwing your overall vdev performance for a bit of a loop. Did you read the thread about using WDIDLE to adjust/disable the spin-down timer on it?

Thanks for the reply. A lot of things seem to be pointing towards some type of networking issue, but my one big problem with that is that the write speed is OK. If it was a network throughput problem, wouldn't it be limiting both paths? Can a different NIC card help solve this (or rule it out)?

Some old parts are from an old gaming rig. I dont want to dip too deep into the piggy bank on a something I am still a total rookie on.

As for the WD Green drive. I wondered if this had an impact as well, so I formatted the NAS, and started over without it included in the pool. Test had no impact to read results.

 

[HoneyBadger]

I'm wondering if it's a poorly implemented or unoptimized driver for the i218V on that board. But yes, the next step in my troubleshooting flowchart would be "try another NIC" - look for one of the older Intel 1000GT/PT/CT cards.

Old parts are fine for learning, I'm just kind of giving the advanced warning if you're planning to put something irreplacable on there. Like I said, the price of admission to "enterprise grade" is a lot lower than you think, especially with older gear being cycled out of datacenters and onto eBay. I've seen dual-socket Dell T610s under US$200.

I think the network issue will mask any potential weirdness introduced by the Green drive. I'd still say the benefits of using it in a 6-drive Z2 outweighs any drawbacks of the mismatch though, rather than a 5-drive Z1.

 

[atlbuckeye]

I'll give another NIC a shot, and report back.

Never crossed my mind to look to ebay for old enterprise grade servers. One of these is what you are talking about?
http://www.ebay.com/itm/DELL-POWERE...933326?hash=item3d2338d98e:g:V-8AAOSwdzVXtcPr

 

[HoneyBadger]

A T610 was basically exactly what I had in mind. Since you've got six drives, the cheaper options of Lenovo TS140s or Dell PowerEdge T110s are off the table. Those come in both 3.5" (LFF) and 2.5" (SFF) format though so make sure to ask if it's the one you want. And for the money that seller's asking I'd expect there to be drive caddies included.

This auction is far from over, but have a look:

http://www.ebay.com/itm/Dell-Powere...928127?hash=item51e9af427f:g:jNwAAOSw0UdXtKKf

Dual hex-cores, triple the RAM, and half a dozen 2TB drives already included.

If you're willing to accept a rackmounted server (a fair bit noisier and a bit more power-hungry) then look at the Dell C2100 or FS12-TY. There's a good posting on this forums called "Ode to the Dell C2100" that summarizes nicely why they're very good boxes for up to 12 drives - but that might be a little bit of overkill.

 

[atlbuckeye]

I know we have derailed from my original issue, but the price of that C2100 is incredibly reasonable. Overkill for only $200-$400 seems pretty acceptable to me. Alas, I would have to buy a server rack as well.

 

[anodos]

I know we have derailed from my original issue, but the price of that C2100 is incredibly reasonable. Overkill for only $200-$400 seems pretty acceptable to me. Alas, I would have to buy a server rack as well.

I fail to see any cons in that series of statements.

 

[HoneyBadger]

Alas, I would have to buy a server rack as well.

IKEA to the rescue!

You may need to go with the longer model with a shelf though.

 

[Mirfster]

I know we have derailed from my original issue, but the price of that C2100 is incredibly reasonable. Overkill for only $200-$400 seems pretty acceptable to me. Alas, I would have to buy a server rack as well.

I fail to see any cons in that series of statements.

Me either. :)

There is a "Turn Key" option from a Seller that is just under $400.00 w/shipping (should be in that thread). I actually purchased one of these personally since I like to "put my money where my mouth is". Thus far I am pleased with the unit with the exception that they forgot to include the fan shroud. I notified them and they FedEx'd it to me and I had it in two days. Need to get other things in order first, but I will do a full write-up/review of it.

 

[atlbuckeye]

Hot diggidy daffodil. 1000GT NIC card yielding me ~23MB/s transfer from the NAS. Thanks Honeybadger for the advice. I was resistant to believe it was a NIC problem given the write speed was so good. This seems like an acceptable level for my everyday operations, and I can tune it from here if I feel the need. 2 MB/s was just unmanageable and I had a feeling something else was amiss.

I couldn't help but laugh at the IKEA solution. I had a feeling you guys would point me at a DIY style rack, but making it out of an end table is pretty ingenious and comical at the same time. Who the heck had that original idea? :)

 

[leenux_tux]

Talking of IKEA..... This web site has a few interesting little projects around a small piece of IKEA furniture. I think it has been mentioned on here previously, if memory serves.

https://wiki.eth0.nl/index.php/LackRack