Now, this is interesting.
I finally received my IBM M1015 SAS controller, so I installed it and flashed the on-board BIOS with the LSI IT firmware.
Using an identical setup from my previous experiments -- I created the following.
HP Microserver N40L
8GB RAM
FreeNAS-8.2.0-BETA3-x64
4 x 2TB drives (SAS Port 0)
2 x 2TB drives (SAS Port 1)
1 x 60Gb SSD (SAS Port 1)
No drives attached to the N40L main board
6 x 2TB drives, formatted 4K (GNOP) -- RAID-Z2
Copying a 15Gb file to an iSCSI device volume resulted in this...
Copying the same file across a CIFS share resulted in this
So... removing the drives from the onboard controller and connecting them to the M1015 controller appears to have removed the 'breathing' network stalling problem that I was seeing before.
Write speed to the RAIDZ2 array also appears to be faster
Code:
ZFS, All drives formatted 4096K (gnop)
ZFS-RAID-2, 6x2TB (gnop) -- New BIOS, IBM M1015 Controller
----------------------------------------------------------------------------------
dd if=/dev/zero of=test.dat bs=2048k count=50k
107374182400 bytes transferred in 528.229085 secs (203272000 bytes/sec) -- 193.86Mb/s
compared to the onboard write speed
Code:
ZFS, All drives formatted 4096K (gnop)
ZFS-RAID-2, 6x2TB (gnop) -- New BIOS with SB settings
----------------------------------------------------------------------------------
dd if=/dev/zero of=test.dat bs=2048k count=50k
107374182400 bytes transferred in 628.539118 secs (170831344 bytes/sec) -- 162.92Mb/s