I am seeing an issue where occasional small file writes cause a much larger disk write, which is tearing through the endurance of my SSDs. This seems to be ZFS committing a mostly empty block transaction to the disk, but regardless of recordsize, a small (i.e. 1KB) file write will cause a 200-400KB transactions.
My question is: is this expected behavior and is not alterable? Or am I missing a setting that would reduce the size of individual transactions?
I'm running FreeNAS-9.10.2-U1. this is a single disk pool with a USB HDD, 4k record size and ashift=12. I see the same behavior on a PCI-e SSD and also a pool of 6x SAS spinning disks behind a PERC H200.
running these two commands in this order, see the iostat output below, the very small dd write produces a very large transaction.
# dd if=/dev/random of=testfile1 bs=512 count=100000
predictably writes 50MB of data to the disk
# dd if=/dev/random of=testfile2 bs=512 count=1
512 bytes end up writing 343KB to the disk
# zpool iostat testusbhdd 1
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
testusbhdd 50.2M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 427 0 50.0M
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 26 0 343K
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
My question is: is this expected behavior and is not alterable? Or am I missing a setting that would reduce the size of individual transactions?
I'm running FreeNAS-9.10.2-U1. this is a single disk pool with a USB HDD, 4k record size and ashift=12. I see the same behavior on a PCI-e SSD and also a pool of 6x SAS spinning disks behind a PERC H200.
running these two commands in this order, see the iostat output below, the very small dd write produces a very large transaction.
# dd if=/dev/random of=testfile1 bs=512 count=100000
predictably writes 50MB of data to the disk
# dd if=/dev/random of=testfile2 bs=512 count=1
512 bytes end up writing 343KB to the disk
# zpool iostat testusbhdd 1
capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
testusbhdd 50.2M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 427 0 50.0M
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 26 0 343K
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0
testusbhdd 100M 3.62T 0 0 0 0