getting slow speeds over gigabit

[digity]

I'm in the middle of moving 15 TB of data from my Synology NAS (DS415+) to my FreeNAS server (11.1U6) and I'm getting only about 300 Mbps (steady). I've always gotten lower than expected transfer speeds with FreeNAS, but never troubleshot it because it didn't serve anything crucial until now. I'm pretty sure the problem isn't the Synology NAS as I've achieved 110+ Mbps transfer speeds with other 1 Gb devices on the network. Based on the environment below any suggestions on getting closer to the expected speeds of gigabit (900+ Mbps)??

Code:

Synology NAS ---[1Gb]---> 24 Port Switch ---[1Gb]---> FreeNAS (Intel 82574L NIC) 
Mix of 12 x 2 TB 7200 RPM SATAIII/6 Gbps HDDs in RAIDZ3
Volume/Vdev created with default options
rsync -auv /PATH/TO/SOURCE USER@1.2.3.4:/PATH/TO/DESTINATION (command running on source machine, Synology NAS)

 

[Elliot Dierksen]

rsynch is never going to give you maximum network throughput because that isn't really what it was designed to do. It is spending a lot of time trying to determine if things need to be copied. If you are trying to do a bulk copy of of data unconditionally, mount the target system share through NFS or SMB and then copy the data. Once it is there, rsynch is a great tool to copy only the things that have changed.

 

[Kennyvb8]

I'm in the middle of moving 15 TB of data from my Synology NAS (DS415+) to my FreeNAS server (11.1U6) and I'm getting only about 300 Mbps (steady). I've always gotten lower than expected transfer speeds with FreeNAS, but never troubleshot it because it didn't serve anything crucial until now. I'm pretty sure the problem isn't the Synology NAS as I've achieved 110+ Mbps transfer speeds with other 1 Gb devices on the network. Based on the environment below any suggestions on getting closer to the expected speeds of gigabit (900+ Mbps)??

Code:

Synology NAS ---[1Gb]---> 24 Port Switch ---[1Gb]---> FreeNAS (Intel 82574L NIC) 
Mix of 12 x 2 TB 7200 RPM SATAIII/6 Gbps HDDs in RAIDZ3
Volume/Vdev created with default options
rsync -auv /PATH/TO/SOURCE USER@1.2.3.4:/PATH/TO/DESTINATION (command running on source machine, Synology NAS)

Test with iperf instead.
Rsync is recursive and slow


Sent from my iPhone using Tapatalk

 

[Chris Moore]

Based on the environment below any suggestions on getting closer to the expected speeds of gigabit (900+ Mbps)??

You might be over-estimating how much speed you should get, but we would need to see some hardware description to be able to decide. It will also fluctuate depending on what you are copying. What are you working with?

Here is a guide to tell you what information you should supply:
https://forums.freenas.org/index.php?threads/updated-forum-rules-8-3-16.45124/

 

[digity]

300 Mbps - rsync ran from SOURCE (rsync -auv /PATH/TO/SOURCE USER@10.0.0.12:/PATH/TO/TARGET)
300 Mbps - scp ran from SOURCE (scp -r /PATH/TO/SOURCE USER@10.0.0.12:/PATH/TO/TARGET)
550 Mbps - scp ran from TARGET (scp -r USER@10.0.0.11:/PATH/TO/SOURCE /PATH/TO/TARGET)
550 Mbps - iperf3, TARGET as server (iperf3 -c 10.0.0.12)

SOURCE = Synology NAS (DS415+), 10.0.0.11
TARGET = FreeNAS 11.1u6, 10.0.0.12

NOTE: had issues getting rsync to run on FreeNAS and also had issues getting iperf3 to connect as client to SOURCE, but I hope the numbers above helps y'all help me.

Supermicro X8DTL-i (rev. 2.x)
Intel XEON X5500 2.6 GHz 4c/8t
24 GB ECC registered memory
2x 64 GB SSD mirrored boot
FreeNAS 11.1u6
LSI 9211-8i (IT flahsed)
Mix of 12 x 2 TB 7200 RPM SATAIII/6 Gbps HDDs in RAIDZ3
Intel 82574L NIC
SMC SMCGS24C-Smart 24 port smart switch

 

[digity]

I'm now on the second half of this project, copying/restoring files back from FreeNAS to the Synology, and I'm getting the same slow speeds (300 Mbps). I tried scp and got the same 300 Mbps. Any ideas?

 

[Chris Moore]

also had issues getting iperf3 to connect as client to SOURCE,

Because iperf3 is the wrong version, too different from the version that is on FreeNAS.

Code:

root@Emily-NAS:~ # iperf --version
iperf version 2.0.10 (11 Aug 2017) pthreads
root@Emily-NAS:~ #

FreeNAS is using iperf version 2, and if I recall correctly, you can use the windows version of iperf that is 2.3 or similar without any difficulty. I have not done it lately.

Mix of 12 x 2 TB 7200 RPM SATAIII/6 Gbps HDDs in RAIDZ3

That is going to be part of the speed problem here. You have all 12 drives in a single vdev at RAIDz3. Now, RAIDz3 isn't horrible, but you only have one vdev which gives you the effective speed of a single drive. It is just how ZFS works. More vdevs gives more speed. The whole reason that I run two vdevs of six drives each is to get the additional performance that more vdevs provides.

 

[Johnnie Black]

FreeNAS includes both iperf v2 (iperf) and v3 (iperf3).

 

[Chris Moore]

FreeNas includes both iperf v2 (iperf) and v3 (iperf3).

Thanks, I didn't realize that.