Write speed = 96MB/second Read speed = 63MB/second

[NorCalTechSupport]

I am new to FreeNAS but I did a lot of homework before ordering any parts so here is my parts list then the description of my write speed issue and hopefully where I should look to make improvements.

1. (SuperMicro) SSG-6047R-E1R24L SuperServer 4U (No RAID card this is a JBOD only)
2. 2ea.Intel Xeon Quad-Core E5-2609V2 2.5GHz Ivy Bridge-E
3. 8ea. 16GB DDR3-1600 1.35V 2Rx4 LP ECC REG
4. AOC-SGP-i4 (4 Port NIC 1GB) in addition to the 4 built in NICs
5. Twenty one Seagate Constellation ES.3 (4TB SAS) 5 year warranty
6. Three 120GB SSD for Cache & logs mounted with Newer Tech AdaptaDrive in 3 adjacent main drive bays
7. Two 1TB WD Red in optional rear facing 2 drive SATA mount (This is for the FreeNAS OS)

I am in testing phase using just eight 4TB SAS drives and the 3 SSDs for my pools. I have a RAIDZ2 shared out as an iSCSI share. I have a ESXi 5.5 VM host running on an SSD with GB NIC and a 2008R2 server located on a FreeNAS iSCSI share. Since I am in testing there is nothing else connected to FreeNAS or ESXi host.

I am in a segregated part of my network so nearly no traffic. I have an Gb 24 port HP Pro Curve PoE swith with

1. Gb line to my network
2. Gb line to VM host
3. Gb line to FreeNAS
4. Gb line to either my PC or Mac.

When I send files from my Windows 7 PC or My MacBook Pro running Mac OS 10.10 to the FreeNAS server I get 96MB/second. My test files are 4,260MB or 4.16GB.

When I send the same files back to the FreeNAS server v. 9.3 I am getting 63MB/second speeds.
My PC has spinning HD at 7,200RPM and my MacBook has a 6G SSD and yet the times are consistently within 1 second of the same?

Any ideas where to look to improve the write speeds would be greatly appreciated & thanks to all.....:)

 

[SweetAndLow]

Your complaining about read speed correct? You last statement didn't make sense because you described a write event twice.

Remove the ssds and what protocol are you using?

 

[gpsguy]

While you're in the testing phase, try striped mirrors instead of RAIDz2. For iSCSI use, it's highly recommended.

 

[mav@]

Can you try to read the same file twice? 100% cache hit on FreeNAS side would help to understand whether problem on storage side or on network side.

 

[NorCalTechSupport]

Thanks for the great replies. I am on travel status for a few days so early next week I will get back into testing and trying out some good ideas I have been hearing and reading.

 

[HoneyBadger]

1. Never do VMFS on parity RAID levels, performance will be terrible. Switch to striped mirrors as suggested by @gpsguy

2. What is the NIC in your desktop Win7 PC and your Macbook? Try reading the same file twice as suggested so that it is pulled from ARC.

3. What model are those SSDs? They may (probably) not be suitable SLOG devices.

4. Speaking of SLOG, iSCSI will not be sync from ESXi unless forced by zfs set sync=always.

 

[NorCalTechSupport]

Thanks for the suggestions. I just used Volume Manager and erased my volume and now it is time for striped mirrors and now I have a different issue?
1. I went to Volume Manage and Manual Setup.
2. I created a 4 drive striped volume as I could not see how to stripe & mirror at the same time when I choose 8 drives.
3. I created a second 4 drive striped volume
4. Now here is the problem, I don't see how to mirror the two striped volumes?

PS I took some other advice and re-installed FreeNAS 9.3 onto two 32GB thumb drives and I am no longer using spinning hard drives for the OS. Thanks everyone for all the great ideas.

 

[gpsguy]

You can't mirror the two striped volumes, though you CAN stripe mirrored volumes.

In Volume Manager, rather than dragging the wheel to the right, drag it down a row, and then across. See the screenshot in message #4 of this thread - https://forums.freenas.org/index.php?threads/raid-10.16910/ It's an older version of FreeNAS, but the concept is the same.

 

[NorCalTechSupport]

Thanks for all the good ideas everybody. Keep the ideas rolling in and I will keep reporting my progress.

I have just finished testing a striped & mirrored 14TB iSCSI share and my speed results were almost identical to the RAID-Z2 share I had last week. The one item in common with both of these read & write tests is I have had 1 or 2 SSDs for cache on both RAIDZ2 (2 SSDs) and striped mirror (1 SSD). I even switched NIC from built in to a four port Gb card and I am within 1% of the same results.

I am currently moving my Server 2012 from FreeNAS to a built in drive on my ESXi 5.5 test server. When I finish I will erase the striped & mirrored share and rebuild it with no SSD as has been suggested and I hope to find a way to speed up my throughput.

Just to be sure that my test Mac & PC were not part of my problem I ran a series of read and write tests between these two machines and my average read and write speed between my test PC & test Mac is 102.7MB/S

FreeNAS accepting files at 95.379MB/S is not bad on a Gb network.
FreeNAS sending files at 65.536MB/S seems a bit slow.

In the below comparison the server is Server 2012R2 located on an iSCSI share on FreeNAS. The PC is i7 with 16GB RAM and Windows 7. The Mac is an i7 with 16GB RAM and Mac OS 10.10.3.

AVERAGE MB/S for Server to PC or Mac = 95.379MB/Second
AVERAGE MB/S for PC or Mac to Server = 65.536MB/Second

 

[Carlos Ramos]

Hello NorCalTechSupport, in your server configuration you use AOC-SGP-i4. How you did? Where are you found the driver to use this on FreeNAS 9.3 ? I am new in FreeNAS, standard installing the network adapter do not works.

Can you help me ? Thank you.

 

[depasseg]

Please start a different thread.

 

[RichTJ99]

Interesting thread - I am curious how the test without the SSDs went? Did the Freenas system performance increase?