Hi everyone,
I recently put together a Supermicro FreeNAS server and the performance is not making much sense to me. Yes, I know having sync on will slow down writes, but the performance is all over the place. My issue is that a single 860 EVO gets the same results as my VM in this giant pool...And the 900p SLOG will do 550k, so I REALLY should be seeing better performance than a single disk. To confuse me even more, Im getting faster writes than reads on the VMs!!! How does that work in a pool of mirror vdevs?? I tried two other 10gb NICS with the same results. Does anyone have any idea as of whats going on or maybe the next step in troubleshooting? I dropped a lot of coin...
Local dd no compression disabled sync: 1.82GB/s write and 2.29G/s read
Local dd no compression sync always: 990MB/s write and 2.7GB/s read.
iSCSI VM (Crystaldiskmark) with compression and sync always sequential: 1200MB/s write and 600MB/s read...???
iSCSI VM (Crystaldiskmark) with compression and sync always 4k q8t8: 295MB/s write and 345MB/s read BOOOOOOOOO
I got similar results with Atto.
I have MPIO/port binding set up in ESXi with RR switching at 1 iop. I had a raidz2 array with better performa
specs as follows:
Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
96GB DDR4 RAM
LSI SAS 9003-8i IT mode
Intel X520 10GB NIC
12x Samsung 860 EVO 2TB SSDs
Intel 900p SLOG
One storage vdev at the painfully recommended 50% pool size.
The main purpose is for a VMware iSCSI datastore.
Sync off
Sync off
=
I recently put together a Supermicro FreeNAS server and the performance is not making much sense to me. Yes, I know having sync on will slow down writes, but the performance is all over the place. My issue is that a single 860 EVO gets the same results as my VM in this giant pool...And the 900p SLOG will do 550k, so I REALLY should be seeing better performance than a single disk. To confuse me even more, Im getting faster writes than reads on the VMs!!! How does that work in a pool of mirror vdevs?? I tried two other 10gb NICS with the same results. Does anyone have any idea as of whats going on or maybe the next step in troubleshooting? I dropped a lot of coin...
Local dd no compression disabled sync: 1.82GB/s write and 2.29G/s read
Local dd no compression sync always: 990MB/s write and 2.7GB/s read.
iSCSI VM (Crystaldiskmark) with compression and sync always sequential: 1200MB/s write and 600MB/s read...???
iSCSI VM (Crystaldiskmark) with compression and sync always 4k q8t8: 295MB/s write and 345MB/s read BOOOOOOOOO
I got similar results with Atto.
I have MPIO/port binding set up in ESXi with RR switching at 1 iop. I had a raidz2 array with better performa
specs as follows:
Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
96GB DDR4 RAM
LSI SAS 9003-8i IT mode
Intel X520 10GB NIC
12x Samsung 860 EVO 2TB SSDs
Intel 900p SLOG
One storage vdev at the painfully recommended 50% pool size.
The main purpose is for a VMware iSCSI datastore.
Sync off
capacity operations bandwidth
pool alloc free read write read write
-------------------------------------- ----- ----- ----- ----- ----- -----
freenas-boot 790M 231G 0 0 0 0
da14p2 790M 231G 0 0 0 0
-------------------------------------- ----- ----- ----- ----- ----- -----
san2vol1 87.5G 12.6T 0 17.4K 0 1.82G
mirror 12.5G 1.80T 0 2.53K 0 267M
gptid/8be06408-2001-11e9-9812-0cc47ad29eba - - 0 527 0 267M
gptid/8c394202-2001-11e9-9812-0cc47ad29eba - - 0 529 0 267M
mirror 12.5G 1.80T 0 2.48K 0 266M
gptid/8c85a54b-2001-11e9-9812-0cc47ad29eba - - 0 511 0 266M
gptid/8ccc22e6-2001-11e9-9812-0cc47ad29eba - - 0 509 0 266M
mirror 12.5G 1.80T 0 2.37K 0 266M
gptid/8d1a76f0-2001-11e9-9812-0cc47ad29eba - - 0 460 0 266M
gptid/8d6220cd-2001-11e9-9812-0cc47ad29eba - - 0 464 0 266M
mirror 12.6G 1.80T 0 2.41K 0 267M
gptid/8db20e7c-2001-11e9-9812-0cc47ad29eba - - 0 527 0 267M
gptid/8dfb3c25-2001-11e9-9812-0cc47ad29eba - - 0 527 0 267M
mirror 12.5G 1.80T 0 2.47K 0 266M
gptid/8e4e60a8-2001-11e9-9812-0cc47ad29eba - - 0 491 0 266M
gptid/8ea62f13-2001-11e9-9812-0cc47ad29eba - - 0 494 0 266M
mirror 12.4G 1.80T 0 2.53K 0 266M
gptid/8f02e24d-2001-11e9-9812-0cc47ad29eba - - 0 518 0 266M
gptid/8f56dc51-2001-11e9-9812-0cc47ad29eba - - 0 519 0 266M
mirror 12.5G 1.80T 0 2.56K 0 265M
gptid/8fbb3155-2001-11e9-9812-0cc47ad29eba - - 0 530 0 266M
gptid/9015ed13-2001-11e9-9812-0cc47ad29eba - - 0 535 0 265M
logs - - - - - -
gptid/907012f3-2001-11e9-9812-0cc47ad29eba 300K 260G 0 0 0 0
Sync off
Code:
capacity operations bandwidth pool alloc free read write read write -------------------------------------- ----- ----- ----- ----- ----- ----- freenas-boot 791M 231G 8 0 22.6K 0 da14p2 791M 231G 8 0 22.6K 0 -------------------------------------- ----- ----- ----- ----- ----- ----- san2vol1 130G 12.6T 18.3K 0 2.29G 0 mirror 18.6G 1.79T 2.62K 0 335M 0 gptid/8be06408-2001-11e9-9812-0cc47ad29eba - - 1.26K 0 171M 0 gptid/8c394202-2001-11e9-9812-0cc47ad29eba - - 1.22K 0 164M 0 mirror 18.5G 1.79T 2.61K 0 334M 0 gptid/8c85a54b-2001-11e9-9812-0cc47ad29eba - - 1.18K 0 160M 0 gptid/8ccc22e6-2001-11e9-9812-0cc47ad29eba - - 1.28K 0 174M 0 mirror 18.6G 1.79T 2.61K 0 334M 0 gptid/8d1a76f0-2001-11e9-9812-0cc47ad29eba - - 1.29K 0 175M 0 gptid/8d6220cd-2001-11e9-9812-0cc47ad29eba - - 1.17K 0 160M 0 mirror 18.7G 1.79T 2.62K 0 335M 0 gptid/8db20e7c-2001-11e9-9812-0cc47ad29eba - - 1.25K 0 171M 0 gptid/8dfb3c25-2001-11e9-9812-0cc47ad29eba - - 1.20K 0 164M 0 mirror 18.6G 1.79T 2.62K 0 335M 0 gptid/8e4e60a8-2001-11e9-9812-0cc47ad29eba - - 1.26K 0 166M 0 gptid/8ea62f13-2001-11e9-9812-0cc47ad29eba - - 1.26K 0 169M 0 mirror 18.6G 1.79T 2.62K 0 335M 0 gptid/8f02e24d-2001-11e9-9812-0cc47ad29eba - - 1.29K 0 175M 0 gptid/8f56dc51-2001-11e9-9812-0cc47ad29eba - - 1.19K 0 160M 0 mirror 18.6G 1.79T 2.62K 0 335M 0 gptid/8fbb3155-2001-11e9-9812-0cc47ad29eba - - 1.28K 0 173M 0 gptid/9015ed13-2001-11e9-9812-0cc47ad29eba - - 1.19K 0 162M 0 logs - - - - - - gptid/907012f3-2001-11e9-9812-0cc47ad29eba 300K 260G 0 0 0 0 -------------------------------------- ----- ----- ----- ----- ----- -----
=
Last edited: