Fan Scripts for Supermicro Boards Using PID Logic 2020-08-20, previous one was missing a file

[StarkJohan]

True. But when the BMC sets everything to 100% because a fan fails we have a problem. A few minutes later when the script notices this, it resets the BMC and sets fans to "script duty cycles", say 25%. Another few minutes later when the BMC finishes its reboot it sets the fan mode to FULL again and fans ramp up to 100%. A few moments later, the script runs and finds the fans are not on the intended RPM and resets the BMC...

Or do you mean that it might just be better to comment out the BMC reset part? Just trying to wrap my head around every possible scenario.

Even when the fan is replaced the loop continues and the script needs to be restarted.

 

[Ericloewe]

It's easy to differentiate some random error from a fan failure. Just a special case for the script to handle.

 

[Glorious1]

Is this not a good way of handling it? Am I missing something?

Frankly, you've thought about this issue more than I have. Have you actually tested pulling a fan live to verify the sequence of events you describe - BMC reset, followed by a cyclic resetting BMC by the script? I don't recall others describing such a loop with recent versions of the scripts.

@Ericloewe, I guess what you're saying is if the script detects a fan missing, it should set the fans at max (in case the BMC didn't) and then terminate?

 

[Ericloewe]

@Ericloewe, I guess what you're saying is if the script detects a fan missing, it should set the fans at max (in case the BMC didn't) and then terminate?

I'm not sure if it's best to let the BMC handle the situation, by letting the script take its hands off until the situation is resolved; or if it should set the fans to 100%. I like the belt and suspenders option, but I'm afraid it might confuse the BMC.

As for terminating the script, that's one option and it makes sense for a lot of people, since many people don't have hot-swap fans and thus need to shut down anyway. I was thinking more along the lines of adding that state to the script, monitoring the situation and reverting to normal after it's over.

 

[StarkJohan]

I "discovered" this as I was curious and pulled one of the fans. I didn't initially realise what had happened but after taking a closer look at the script I then confirmed the behaviour described in post #161 twice.

Maybe the best route to go in this case is for the script to recognise that most fans suddenly are at max, check if any of the fans are "missing", reporting this and then just monitor until the fan is swapped as Ericloewe suggests.

Btw, I changed the post init to make use of "screen" as this makes it easier, at least to me, to "check in" on the script.

Code:

/usr/local/bin/screen -dmS spinpid2 /path/to/spinpid2.sh

 

[Glorious1]

I wonder if the best approach is the simplest: remove the ability of the script to reset the BMC at all. It seems like the BMC will reset itself as needed. If a fan goes missing, the script would still control the other fans, increasing duty cycle as needed.

 

[Kevin Horton]

I wonder if the best approach is the simplest: remove the ability of the script to reset the BMC at all. It seems like the BMC will reset itself as needed. If a fan goes missing, the script would still control the other fans, increasing duty cycle as needed.

I'm running a different PID control script, but I've seen a small number of cases on one of my servers where the BMC froze, which was detected by the script, triggering a BMC reset which then restored normal operation. So, it is useful to have the script do a BMC reset if needed, at least on one of my servers.

 

[StarkJohan]

As long as the script differentiates between one or more fans failing/missing and a completely unresponsive BMC that might be the best solution.

 

[Glorious1]

How would you test for a completely unresponsive BMC?

 

[StarkJohan]

I don't know, but maybe assuming a frozen BMC if no sensor values are returned might be ok?

 

[charlie89]

Doooohhh! Thanks.

My point still stands, though -- we can't adjust the PWM on X9-series Supermicro mobos.

Just as a side note for all who use an X9 Supermicro board with an onboard Nuvoton WPCM450 BMC chip: I have posted a fan script here -> https://forums.freenas.org/index.php?threads/fan-script-for-x9-motherboards.69550/#post-481280

 

[Stux]

My script checks for a BMC which is not respecting the PWM setting.

Hence the reset loop.

The real question is does the BMC stop forcing high once the fan is restored?

Btw you can test the by simply unplugging a fan. It doesn’t have to be a hot swap fan.

 

[Ericloewe]

I think it does. I seem to recall having accidentally unplugged a fan with the server running.

That was not the same time my thumb was violently mangled by a 3k RPM Noctua NF-F12. To prove that I didn't learn anything, it was after said incident.

 

[StarkJohan]

The real question is does the BMC stop forcing high once the fan is restored?

Yes it does. I commented out the BMC reset as suggested for now which works as fine as expected.

 

[Stux]

Yes it does. I commented out the BMC reset as suggested for now which works as fine as expected.

The problem is when the BMC hangs and the fans are low. Might want to ensure that can’t happen to avoid damage.

 

[StarkJohan]

Might want to ensure that can’t happen to avoid damage.

You're absolutely right. I have no clue on how to test for a frozen BMC "directly"though. Your approach seems reasonable and perhaps could be implemented.

 

[Platter7]

I have a new rig with an x11ssh-f mobo and 4 noctua fans. I would like to use these scripts for optimal control of the fans...
But I am completely new in the use of scripts. I know how to connect with putty … but then?? Where and how do I copy this code... how to let it automaticly be executed at startup?
Question: where can I find a good resource on this matter?

 

[Glorious1]

First read everything in the resource post. Put the script in your user directory on the FreeNAS machine. Change the file extension to sh. Use chmod to make it executable. Use the FreeNAS webGUI to set it as an post-init task to start after booting.

 

[Platter7]

First read everything in the resource post. Put the script in your user directory on the FreeNAS machine. Change the file extension to sh. Use chmod to make it executable. Use the FreeNAS webGUI to set it as an post-init task to start after booting.

Ok. Did al that. I've got the script working from the command line, and it works like a charm. But I think it doesn't start at startup. I have located the script the root. under /usr/home/…/spinpid2.sh
Starting a script Post Init from the root in de GUI does not seem possible, so I chose as type "command" post init: /usr/home/../spinpid2.sh but it won't start. Maybe I should put the file in a different directory?

 

[Glorious1]

Ok. Did al that. . . . but it won't start. Maybe I should put the file in a different directory?

As suggested, put the script in your user directory. Other locations might work, but I suggested that because I know it works. Then, in my case, the post-init line is
/mnt/Ark/Jim/bin/spinpid.sh. Type should be "script", not command.