Truenas 12 or older vs 13 NFS speed

[remonv76]

Hi all,

So we are doing some testing with Truenas and stumbled on a weird issue. Somehow Truenas 13 is much, much faster then the older versions and we can not figure out why this is. We did some testing and around 128k the disk speed flattens around 60-70MB/s. But with Truenas 13, we are getting around 230MB/s.

Setup and configuration are exactly the same. Basic NFSv3 config, RAIDZ, RAIDZ2 or MIRROR, it all does not matter. For testing we did NOT include a SLOG or L2ARC. Just plain RAIDZ, RAIDZ2 or MIRROR.

Pool: 16x 1.8TB SAS disks (4kn/512e) - ashift12
Network Truenas (11.3, 12.0.U8, 13.0) 2x 40 Gb/s in failover
NFS datastore

Esxi host 5.5, 6.5 or 6.7 with 2x 10Gb/s in failover (all ESXi versions gave the same performance issue with Truenas 12 or older)
Windows 10 client running atto disk benchmark and CrystalDiskMark

 

[Volts]

Are you asking "is everything working correctly and safely?"
or "what improvements in 13 contribute to this speed increase?" ?

 

[remonv76]

Well definately the last question. How come Truenas 13 is so much faster or why is 12 and older limited? So the question is, what are we seeying here?

Correctly and safely, probably not, because 13 is still in the early stages. We are waiting for U1 At the end of this month.

 

[remonv76]

As an update, we have tried every possible combination on Truenas 12 to get the same speeds as 13. network performance is the same as 13. Even with iscsi we get the same performance difference. This is so weird!
And we need a good baseline performance for the pool, before using this storage unit in production. That’s why we decided to wait for U1, instead of using the stable production versions 11 or 12.

 

[anodos]

In 13 we bumped the default count of NFS servers from 4 to 16 (which is more appropriate on most modern hardware). (In addition to general OS and filesystem improvements).