SOLVED NVME Trouble? Crashed Server?

[kspare]

Last night one of our storage servers crashed. The daily security run output has the following:

nvme0: DATASET MANAGEMENT sqid:1 cid:63 nsid:1
> nvme0: LBA OUT OF RANGE (00/80) sqid:1 cid:63 cdw0:0
> nvme0: DATASET MANAGEMENT sqid:1 cid:62 nsid:1

The NVME's are Intel P3700 400GB, OP to 20GB and formatted for 4K.

Just wondering if anyone has run into this?

 

[dlavigne]

Which FreeNAS version?

 

[kspare]

11.2U5. I figured it out though. One card didn't have the latest firmware on it. I could get it to create these errors simply by added and removing the nvme card as my L2 Arc cache. After the update it all went away and performance actually went up. Great results!