Hundreds of Fan FANA Deasserted errors

[Sawtaytoes]

After upgrading to Bluefin 22.12.1, I started getting all these Fan FANA Deasserted errors.

OS Version: TrueNAS-SCALE-22.12.1
Product: Super Server
Model: AMD EPYC 7252 8-Core Processor
Memory: 252 GiB

This is after 5 minutes:

WARNING​

Fan FANA Deasserted Lower Critical going low : Reading 1540 < Threshold 420 RPM.​

2023-03-08 07:36:21 (America/Chicago)
Dismiss
error

WARNING​

Fan FANA Deasserted Lower Non-recoverable going low : Reading 1540 < Threshold 280 RPM.​

2023-03-08 07:36:21 (America/Chicago)
Dismiss
error

 

[ummario]

I got the same problem. Never saw this before

 

[sretalla]

On face value, it's saying your fan is going too slow, so it's trying to take it up to the next step of speed...

The issue is that the current reading looks higher than the threshold, not lower than it, so it's nonsense.

Can you see if the fan connected to that header is actually spinning?

You may need to play with some of the ipmitool commands to change the thresholds and/or set the actual fan mode/speed to something else.

 

[neofusion]

Please correct me if I'm wrong, but doesn't Asserted mean something is wrong and Deasserted mean it's working correctly again?
Something is making your fan spin slower than the threshold the warning is tied to triggering the Assertion, temporarily at least.

 

[Sawtaytoes]

All my fans are spinning. Two of them are Noctuas. I can't remember if I slowed them down in the UEFI or with one of those adapters, but I know I made them spin slower somehow.

 

[SeiyaGame]

You can find documentation on Supermicro's IPMI with Noctua's FAN here :

• https://blog.pcfe.net/hugo/posts/2018-08-14-epyc-ipmi-fans/
• https://forums.servethehome.com/index.php?resources/supermicro-x9-x10-x11-fan-speed-control.20/

I have a Supermicro H11SSL-i and here is what you need to know :

• FAN1 to FAN5 is CPU zone
• FANA et FANB is Perpipheral zone

The first thing to do is to change the thresholds of your fans into errors.
For example, the speed of all my fans is between 300rpm and 1500rpm, so we need to set these thresholds:

for i in 1 2 3 4 5 6 A B; do ipmitool sensor thresh FAN${i} lower 0 100 200;done
for i in 1 2 3 4 5 6 A B; do ipmitool sensor thresh FAN${i} upper 1700 1800 1900;done

Then, if the fan speed does not slow down or is not taken into account, you must do a hot reboot of the IPMI :

ipmitool bmc reset warm

Finally, if you want to control the actual speed of your fans, you can set the duty cycle: (Need to redo it, every restart)

# Set CPU zone (0) to 40%:
ipmitool raw 0x30 0x70 0x66 0x01 0x00 0x28

# Set Perpipheral zone (1) to 60%:
ipmitool raw 0x30 0x70 0x66 0x01 0x01 0x3C

I hope it's help you

 

[picklefish]

Tons of people seeing it over in https://www.truenas.com/community/threads/update-causing-fan-errors.108276/page-2 something changed with TrueNAS

 

[indivision]

It looks to me like TrueNAS was updated to include these errors in the GUI which is a good thing.

The people seeing it for the first time probably didn't realize they had errors happening in IPMI all this time and need to fix their fan thresholds.

 

[Ericloewe]

Yeah, let's cut down on the number of threads on this topic. Please continue here:

Update causing Fan errors

I have just updated to TTrueNAS-13.0-U4 and I am getting these errors: New alerts: * Fan FAN 4 Deasserted Lower Non-critical going low : Reading 750 < Threshold 600 RPM. New alerts: * Fan FAN 4 Asserted Lower Non-critical going low : Reading 600 < Threshold 600 RPM. Current alerts: * Fan FAN 4...

www.truenas.com