I threw together a rig with FreeNAS (latest version) to benchmark a SAS 6Gb/s enclosure I just got, EMC KTN STL3, and the hard drives that it came with; 15 x 600 GB SAS 15k RPM - mixed sizes, brands and SAS versions (drive details below). The drives showed up as multipath disks, I threw all 15 drives in a stripe/RAID0 pool and disabled sync and compression. I used dd to do the benchmark (commands used below), because I couldn't I don't know how do make jails yet so I can use bonnie++. I averaged 1261 MB/s (9.9 Gb/s) for write speeds and about the same for read speeds. Does this sound about right?
I ask because before setting these drives up in FreeNAS, I tested individual drive performance first with ATTO Disk Benchmark (Windows) and got 200 MB/s for the 1 x 3.5" HGST HDD, averaged 222 MB/s for each of the 6 x 2.5" Toshiba HDDs and averaged 249 MB/s for each of the 8 x 2.5" HGST HDDs (for both read and write for all). That's 3524 MB/s (27.5 Gb/s) total and while I understand I won't get that exactly in stripe/RAID0 in FreeNAS I'd imagine I'd get way more than 1261 MB/s (9.9 Gb/s).
Server Config
Gigabyte GA-X79-UP4 mobo, 1 x XEON E5-2620, 8GB DDR3, LSI 9201-16e
SAS schema
LSI 9201-16e 6Gb/s SAS PCI-e x8 HBA (IT mode) in PCI-e x8 mobo expansion slot <-- 2 x SFF-8088 1 meter cable (for multipath SAS) --> connected to both controllers on the EMC KTN STL3 (the ports labeled with two circles)
Drives
Am I getting the best performance here or can I tweak something to get better speeds?
P.S. - For the random tests I always get 89 MB/s (0.7 Gb/s), even for other tests with other types of disks and RAID sets (SSDs, SATA 7.2k RPM HDDs, etc.), so I didn't include those results as I believe something's wrong with how I'm implementing that test.
I ask because before setting these drives up in FreeNAS, I tested individual drive performance first with ATTO Disk Benchmark (Windows) and got 200 MB/s for the 1 x 3.5" HGST HDD, averaged 222 MB/s for each of the 6 x 2.5" Toshiba HDDs and averaged 249 MB/s for each of the 8 x 2.5" HGST HDDs (for both read and write for all). That's 3524 MB/s (27.5 Gb/s) total and while I understand I won't get that exactly in stripe/RAID0 in FreeNAS I'd imagine I'd get way more than 1261 MB/s (9.9 Gb/s).
Server Config
Gigabyte GA-X79-UP4 mobo, 1 x XEON E5-2620, 8GB DDR3, LSI 9201-16e
SAS schema
LSI 9201-16e 6Gb/s SAS PCI-e x8 HBA (IT mode) in PCI-e x8 mobo expansion slot <-- 2 x SFF-8088 1 meter cable (for multipath SAS) --> connected to both controllers on the EMC KTN STL3 (the ports labeled with two circles)
Drives
- 8 x 2.5" HGST SAS 12 Gb/s 15k RPM HDD
- 6 x 2.5" Toshiba SAS 6Gb/s 15k RPM HDD
- 1 x 3.5" HGST SAS 6Gb/s 15k RPM HDD
Code:
dd if=/dev/zero of=test.dat bs=2048k count=10000 dd of=/dev/null if=test.dat bs=2048k count=10000 dd if=/dev/random of=test.dat bs=2048k count=10000
Am I getting the best performance here or can I tweak something to get better speeds?
P.S. - For the random tests I always get 89 MB/s (0.7 Gb/s), even for other tests with other types of disks and RAID sets (SSDs, SATA 7.2k RPM HDDs, etc.), so I didn't include those results as I believe something's wrong with how I'm implementing that test.