Slow NFS in FreeNAS VM

[nasathome]

I built a new FreeNAS file server using 11.2, and so far the build has gone very smoothly.

I have 2 pools - One on a pair of 6TB disks providing datasets that are shared on my network via CIFS and NFS, and a second fast pool on a pair of Samsung EVO SSDs to store virtual machines. I have a couple of jails (including the Plex plugin), and a couple of VMs running CentOS 7.

On the same network I have a physical CentOS 7 linux system (that I'm trying to replace with the VM). The family also has a bunch of windows and Mac laptops/desktops that use the shared storage from FreeNAS. Performance has been great serving data to other computers on the network - it can easily flood the 1 Gb/s network, Plex works great, etc.

So - here is my problem...

NFS mounts to FreeNAS are very slow on the linux VM running on FreeNAS using bhyve. The physical linux machine can saturate the 1 Gb/s NIC with NFS copies, but the copies to the same mounts on the VM are extremely slow. The setup, fstab options, etc, are identical between the VM and physical hosts.

To rule out general networking issues, I ran iperf tests, and get 0.94 Gbps between the physical linux machine and either FreeNAS or the linux VM (in both directions), and over 2 Gbps between FreeNAS and the VM (again, in both directions). So I don't think this is a networking issue.

The bhyve VM is using VirtIO disk to the SSD pool, 8GB RAM, 4 CPU, NIC is Intel e82545 (e1000).

Is anyone else seeing NFS issues in Linux VMs? Any thoughts on how to troubleshoot this?

Thanks.

 

[nasathome]

Sometimes taking the time to detail your problem for others leads you to see rocks you haven't overturned...

I changed the NIC from e1000 to VirtIO, and my iperf performance between the VM and FreeNAS went up to nearly 5 Gbps, and NFS performance jumped from about 10 Mbps to about 100 Mbps, which nearly matches the 115 Mbps of the physical host over a 1 Gbps link. I would expect it to drastically outperform the physical host which has the gigabit NIC bottleneck, but it is at least a workable system this way.

I'm still curious if anyone has thoughts on what might be limiting the throughput and how to debug it.

 

[nasathome]

I spoke too soon. It's definitely faster with the VirtIO NIC - but only by about 2X, which matches the iperf improvement. I'm not sure where the 100 Mbps result came from, I can't seem to repeat it. So I'm still 3X or 4X slower over NFS on the VM compared to a physical machine.

 

[vinistois]

Have you gotten anywhere with this?