Slow file transfer speed

[T3mpar]

Hi All,

I'll cut to the point - getting 10MB transfer speed from a wired ethernet Windows machine to a wired ethernet TrueNAS box.

iperf3 test shows that the link speed between the two devices is gigabit with a transfer speed of ~910mbps.

I have a SMB share setup on the TrueNAS machine which I am trying to transfer files to from the Windows machine.

It's not the cable otherwise iperf wouldn't be saying gig link, it's not the switch for the same reason.. which leads me to believe it's maybe the HDD speed? But this seems low even for that?

• Motherboard make and model - Gigabyte GA-B85M-DS3H-A Micro ATX LGA1150 Motherboard
• CPU make and model - I7-4790
• RAM quantity - 16gb
• Hard drives, quantity, model numbers, and RAID configuration, including boot drives - 256gb SSD boot, 2x 4TB HDD (Raidz1)
• Network cards - Motherboard gigabit NIC

 

[HoneyBadger]

The GA-B85M-DS3H-A uses an onboard Realtek network controller, which has been shown to have poorer than expected transfer speeds under real-world conditions.

Can you also provide the model numbers of your 4TB HDDs? Shingled Magnetic Recording (SMR) could potentially be in play which is also known to adversely impact transfer speeds.

 

[T3mpar]

Thanks for the reply, the HDD are both 4tb Seagate Barracuda drives.

It’s odd because when I do a FTP transfer, I get the full 920mbps transfer speed. When I do a SMB transfer (to the same folder in the same pool ect ect, the exact same place) I get capped at about 100-110mbps.

This leads me to believe that the network connection is fine? Again, as iperf shows the connection being around that 920mbps speed which I can achieve on a FTP transfer. Appears the issue is with SMB?

 

[sretalla]

the HDD are both 4tb Seagate Barracuda drives

And those are typically SMR... bad news for ZFS.

 

[T3mpar]

And those are typically SMR... bad news for ZFS.

Any reason why FTP vs SMB transfers have such wildly different speeds though? Feel like if it was a drive issue then neither of these would be able to utilise the network bandwidth and would be bottlenecked by the drive speed?

 

[joeschmuck]

Any reason why FTP vs SMB transfers have such wildly different speeds though?

They are different protocols is all I can say. Also your Windoze machine could be adding to the issue, it may not be just the TrueNAS machine alone. The RealTek NIC thing is a real issue, I didn't think so myself but found out the hard way. I can't say it dropped my speeds to 10MB but I did loose almost half of my speed during testing.

Lastly, you wee asked for the HDD model numbers and didn't provide it. So maybe those are SMR drives, maybe not. If you do provide the model numbers then we can point a finger or not.

It’s odd because when I do a FTP transfer, I get the full 920mbps transfer speed. When I do a SMB transfer (to the same folder in the same pool ect ect, the exact same place) I get capped at about 100-110mbps.

You will need to be very specific in how you conducted the tests. Did you reboot both systems after each test (read or write transfers)? Were the tests conducted exactly the same except for the protocol? What tool did you use to transfer the data? Did you transfer files to/from the NAS, file sizes, total size? And is it repeatable (I suspect it is). My point is, throughput testing needs to be exact. Also, is compression turned on for the dataset? I like to turn off compression as it provides a realistic and consistent data point.

You should read through the forums as there are a lot of posting about SMB vs. FTP speed differences. But I'm sure you will figure out the issue, and it could be as simple as the NIC.

 

[LarsR]

The Datasheet for all baracuda Drives Show that they're all smr drives.
So your Speed should be the accumulation of the crappy realtek nic and smr drives...

Edit: The 500Gb and 1Tb are CMR

 

[T3mpar]

They are different protocols is all I can say. Also your Windoze machine could be adding to the issue, it may not be just the TrueNAS machine alone. The RealTek NIC thing is a real issue, I didn't think so myself but found out the hard way. I can't say it dropped my speeds to 10MB but I did loose almost half of my speed during testing.

Lastly, you wee asked for the HDD model numbers and didn't provide it. So maybe those are SMR drives, maybe not. If you do provide the model numbers then we can point a finger or not.


You will need to be very specific in how you conducted the tests. Did you reboot both systems after each test (read or write transfers)? Were the tests conducted exactly the same except for the protocol? What tool did you use to transfer the data? Did you transfer files to/from the NAS, file sizes, total size? And is it repeatable (I suspect it is). My point is, throughput testing needs to be exact. Also, is compression turned on for the dataset? I like to turn off compression as it provides a realistic and consistent data point.

You should read through the forums as there are a lot of posting about SMB vs. FTP speed differences. But I'm sure you will figure out the issue, and it could be as simple as the NIC.

I’d understand the drives or NIC being at fault if in all tests I conducted the speed was the same but it’s not? Maybe my explanation next will shed some light?

I used filezilla and windows file manager to conduct the FTP and SMB tests respectively. Both were write tests using the same file (a 4GB single file movie). I had a 1GB Ethernet connection between the two machines as proven by iperf3 tests both times. LZ3 compression was enabled in both scenarios. I further tested using commandline copy and robocopy for SMB yielding the same results I am always capped at 10MBps or 100mbps.

The only thing I can think of that maybe would affect it is my network topology but this further makes no sense. My connection to the internet (WAN) is around 120mbps to the TrueNAS box, however my LAN connection between the two machines is gigabit. Thats the only correlation between the speeds I am seeing vs the expected speed?

Maybe I just stick to using FTP as I know it will result in the full gigabit read/writes it’s just very odd behaviour given that all hardware is the same and the only difference is the protocol and software?

I am going to add a dedicated gigabit Ethernet card just to be sure, anyone got any links to ones they’d recommend? (Feel like this is already a forum post, excuse my laziness if anyone could link me that way..)

 

[joeschmuck]

Did you look at the recommended Hardware Guide, page 13 to be specific? Look in Resources section on the left pane. Download the pdf and give it a read.

As for the speed testing you have done, the file being transferred is too small, it can easily fit into the RAM cache. Filezilla while a nice piece of software is not a benchmark test. A better test I only use as a generalized test is CrystalDiskMark. Setup the file size to 32GB (it's a bit overkill but it will rule out a cache interference) and set the test to run 3 to 5 times using random data (random data is the default). Also, you should use a dataset that has compression disabled (enabled by default). It will take several minutes to run, and then once done, post a screen shot. As you troubleshoot the problem, use this same test to test out the changes.

The GA-B85M-DS3H-A uses an onboard Realtek network controller, which has been shown to have poorer than expected transfer speeds under real-world conditions.

Odds are this is your problem and yes, Realtek NICs can kill the transfer rates. Realtek NICs are not well supported in BSD or Linux as Windows is.

Can you post the output of zpool status and zpool list and that too might glean some extra information that might help.

Last thing for this posting, your Windoze PC, what type of NIC does it have? Have you tried a direct Ethernet cable connection between the NAS and PC? I actually don't doubt that you have a 1000 mb link, but ruling out hardware is generally an easy thing to do.

SMR drives will have a negative effect once they have been completely written to once, the Shingled part kills write speeds after that point.