SLOG not changing disk performance of volume during testing

[Eds89]

God I don't get it lol.
Got home and after powering up the host, and running the same tests again on my original volume that was 32k non sparse and was previously generating 450MB/s, it now gives me 900MB/s reads, but now 1000Mb/s writes
I dropped the volume back down to a 4 disk mirror vdev from the previous 6 disk stripe, and see the same 900MB/s reads, 1000MB/s writes.

The only thing I can think of that I changed before a reboot, would be the MTU setting on the storage interface in FreeNAS.

 

[Eds89]

BINGO!!!!

Now have my SSD SLOG added to my volume and a zvol with sync=always set.
CrystalDiskMark now shows about 850 - 900MB/s reads, and a lovely ~700MB/s writes ;)

Happy days! Thanks for all the input everyone, think I have my setup!

 

[kdragon75]

If 1GB can fit in my cache, then surely the write speed reported should be higher than 450MB/s, as it should be written to RAM quicker than that

It is higher that 450 when you use the 64k chunks, in fact you hit over 1GB at a queue depth of 32 on one thread.

Ok, so if 32k maxes writes at 450MB/s, the only reason that writes are faster when still at 32k but sparse, is because ESXi has less to do? If my 32k sparse and 64k sparse/non-sparse all produce the same results, is it better to stick to 32k for better IOPs?

You don't get to pick you Io size outside of your bench mark. It's just whatever the IO happens to be. If I request to read a small file, I only get that file (even if the disk has to read the whole sector on the media). If I request a large file, each IO will be larger.
As for the sparse, ESXi has almost nothing to do and there is less real data to transfer. This all helps the apparent performance.

Keep in mind unless you tell crystalmark otherwise, it requests all write as sync. Now that sync gets lost over iSCSI (For some unfathomable reason) so your still limited to the achievable network speed as the write still needs to get through the drivers to get to the cache (Your FreeNAS TXGs).

Hmm... Odd it's so much better after a reboot. As long as you can still see the writes hitting the ZIL on the SLOG, you should be safe and all set.