Hmmm ... for me that was precisely what I expected.
1st it is inherited like almost any other ZFS attribute.
2nd the docs say:
This value represents the threshold block size for including small file blocks into the special allocation class. Blocks smaller than or equal to this value will be assigned to the special allocation class while greater blocks will be assigned to the regular class.
So if I set special_small_blocks=4k, I fully expect all blocks <= 4k to end up on the special vdev.