SOLVED Tuning SMB for better read performance (NVMe drive)

[lonelyzinc]

Why is SMB so slow by default on FreeNAS? I'm testing from a standalone Samsung 970 NVMe drive that is capable of reading at over 2400 MB/s, to eliminate the disk speed as a factor. Using AJA System Test and CrystalDiskMark.

The drive shared out over AFP will get 825 MB/s reads, but only 350 MB/s read with SMB on the same macOS 10.15.1 machine. On Windows 10 it's even slower, at 1045 MB/s WRITE, but only 256 MB/s read.

Is there an Auxiliary Parameter I need to set for SMB to read fast?

 

[anodos]

Samba's AIO is actually only turned on for reads. You can turn off AIO for reads by setting "aio read size = 0".

 

[lonelyzinc]

Samba's AIO is actually only turned on for reads. You can turn off AIO for reads by setting "aio read size = 0".

Having that on slightly improved SMB read performance in benchmarks, but it's still half the speed of AFP.

 

[seanm]

Samba's AIO is actually only turned on for reads. You can turn off AIO for reads by setting "aio read size = 0".

So wait, things are faster when async is off?

 

[anodos]

So wait, things are faster when async is off?

A _benchmark_ might be faster with AIO off.

 

[lonelyzinc]

Clearly it's something with the default SMB configuration. There's gotta be an smb option or something to greatly improve it.

I put a Samsung 970 NVMe drive in the FreeNAS box, and it's still reading 350 MB/s over SMB. The NVMe drive itself can read at like 2400 MB/s. Over AFP the same drive will get 825 MB/s in AJA.

I'm seeing similar issues on Windows 10 using CrystalDiskMark: 263 MB/s read and 1049 MB/s write to this SSD.

 

[anodos]

Clearly it's something with the default SMB configuration. There's gotta be an smb option or something to greatly improve it.

I put a Samsung 970 NVMe drive in the FreeNAS box, and it's still reading 350 MB/s over SMB. The NVMe drive itself can read at like 2400 MB/s. Over AFP the same drive will get 825 MB/s in AJA.

UPDATE: According to the research I had done, I thought SMB signing was OFF by default on macOS 10.15, but in my case it wasn't. Disabling SMB signing on my Mac client improved the read speeds into the 500-550 MB/s range. I guess this is as good as it get's for SMB on Mac?

Perhaps retry with 11.3 Beta 1.

 

[lonelyzinc]

Perhaps retry with 11.3 Beta 1.

I figured it out, all I needed to do was to manually set the MTU to 9198 on my FreeNAS and macOS client (default value on my 10G switch). When you don't set this manually it will use the default of 1500.

No tuning to the Samba config on FreeNAS required, although I did make sure SMB signing was OFF on my Mac client. After that the SMB read speeds went waaaaay up. Very happy now!! (see attachment)