NFS slow with vmware ESX and ESXi

[leonroy]

I have three NFS servers hooked up to vSphere. Both Linux servers give me 60MB/s and above performance. FreeNAS gives only 3MB/s.

Mounting the FreeNAS share on another Linux box gives throughput as high as 90-100MB/s. It seems that ESX always mounts NFS shares as sync. Despite selecting NFS Asynchronous Mode in FreeNAS I'm still seeing poor performance.

Any suggestions?

 

[roderick1st]

I have exactly the same issue. If I mount the dataset via iscsi I get 70+ MB/s. Yet if I mount the same dataset via NFS I get 3MB/s.

Any ideas?

 

[jgreco]

Add Tunable:
Variable: vfs.zfs.zil_disable
Value:1

If that works, REMOVE the setting, it's dangerous (as in you should expect to lose data eventually). But it's an easy test to see if you're just being hit by sync write issues in ZFS, which you probably are, which is why a SSD for ZIL is advisable.

And read what I said again: this setting is not a fix, even if it appears to be.

 

[scotch_tape]

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut at rutrum mauris, a bibendum nisl. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Duis gravida odio in eros porta porttitor. Proin ornare tincidunt mattis. Vestibulum vel nisl dignissim, consectetur felis sed, tristique augue. Nulla semper elit nec sem pulvinar, non convallis velit tincidunt. Nam egestas accumsan quam. Mauris efficitur nisi eget tellus suscipit, vitae posuere lacus suscipit.

 

[jgreco]

The tunable isn't particularly dangerous to try for a few minutes as long as the ZFS system doesn't lose power or crash or something. What's more likely at risk is any data stored on the filesystem, specifically anything that gets modified.

The current way in ZFSv28 to do this is "zfs set sync=standard poolname" (enable normal ZIL operations, i.e. normal mode) or "zfs set sync=disabled poolname" (the risky setting).