Effects of different MTU values on ISCSI performance
I did some testing last night using 2 FreeNAS ISCSI targets and a Windows 7 initiator. Here is what I found:
The FreeNas 8.0.2 (64Bit) machine is running an AMD64 3500+ (2.2Ghz) with 2GB of memory and an Intel PCI gigabit NIC (EXPI9301CTBLK - Gigabit CT).
Target # 1 - Two WD Blue 500GB drives (WD5000AAKS) RAID 0 stripe on LSI SAS 3041E-R
Target # 2 - Single Samsung 320GB drive (HD321KJ) on LSI SAS 3041E-R
The ISCSI initiator is a Windows 7 Enterprise (64Bit) machine running an Intel Pentium E5700 (Dual Core) 3.0Ghz with 12GB of memory and networked via an Intel PCI express Gigabit NIC (PWLA8391GT - Pro 1000 GT).
Targets and Initiator are connected using CAT5 direct cabling (no switch used). Full Duplex Mode enabled and verified on both systems (using ifconfig and system event viewer respectively).
First for a DAS comparision, when measured locally using the Gnome disk utility, the drive array used in Target #1 boasts read speed of 198MB/s. I cannot test it at this time, but I recall the write speeds are similar, except a little bit faster. The singe drive used for Target # 2 produces read speeds of 51MB/s and I suspect the write speeds are similar. I have not tested I/O either locally or remotely on either drive.
Tested the following read speeds using HDTUNE 2.55 using a 64KB block size.
Target #1
WIN7 MTU 1500, FreeNAS MTU 1500 - 31.5 MB/s
WIN7 MTU 1500, FreeNAS MTU 9000 - 22.0 MB/s
WIN7 MTU 9014, FreeNAS MTU 9000 - 11.6 MB/s
WIN7 MTU 9014, FreeNAS MTU 9014 - 33.8 MB/s
Target #2
WIN7 MTU 1500, FreeNAS MTU 1500 - 30.6 MB/s
WIN7 MTU 1500, FreeNAS MTU 9000 - 31.2 MB/s
WIN7 MTU 9014, FreeNAS MTU 9000 - 30.8 MB/s
WIN7 MTU 9014, FreeNAS MTU 9014 - 31.4 MB/s
Observations and conclusion
While Target # 2 was barely effected by MTU mismatches, Target # 1 suffered severe performance degration. Additionally, when Jumbo Frames were correctly matched, neither target improved much versus the default MTU of 1500.
Interestingly, the results suggest packet size was not the limiting factor in this particular environment. This is especially true because Gigabit Ethernet is supposed to be capable of data throughput approaching 125MB/s.
CPU utilization also was not the limiting factor as neither machine saw CPU rates exceed 10-25%.
Perhaps in an environment where CPU utilization or packet size is the limiting factor, there may be some noticeable improvements when enabling correctly matched Jumbo Frames. In this case however, it would seem the limiting factor is FreeNAS 8.0.2. Anecdotal evidence scattered on the Internet suggests this to be the situation.
While I think FreeNAS has a very pretty and polished web interface, I have also found this NAS product to be a bit flakey. Most changes require a reboot to before they take effect. Almost like Windows in that regard. I will be moving on to something else....Solaris 11, Linux, or any one of the other NAS oriented products such as Open-E Data Storage Software V6 Lite.