Fan Scripts for Supermicro Boards Using PID Logic 2020-08-20, previous one was missing a file

[fta]

The code as written iterates the duty cycle to the correct place to have the average temperature equal the target. I've never seen a need to try it with Ki equal to anything except zero, so I guess it is more of a PD controller than a PID controller.

I thought I'd try to be helpful so this implementation could be fixed (should be pretty simple) so it works with the integral turned on, but...

Let us know how your version works, and how the performance compares to the version described by @Glorious1 .

the first response is passive aggressive. I had heard the freenas forums had gotten better. Guess not.

 

[Ericloewe]

Now that this is mentioned, doing curr_dc = curr_dc + ... is, in effect, an integration, which doesn't make much sense when used like this, since the result is a controller with a double integration, single integration and simple gain. Cut out the double integration since K_i is zero and it's a PI controller with a fairly convoluted description.

Generically, integral control is slow but eliminates steady-state errors and derivative control responds more quickly to large changes while dealing poorly with slowly-accumulating errors.

 

[fta]

Now that this is mentioned, doing curr_dc = curr_dc + ... is, in effect, an integration,

This is a good point, although there is a correct way to do PID with the integral in that form (see below). My guess is the author intended to use the standard form of the PID algorithm but since Wikipedia and the like usually list the algorithm with a bias of 0 (i.e. they don't list it at all), the author didn't know to include the bias term. In a process where the control can be fully shut off, a bias of 0 is ok, but in our case, we have a minimum fan speed and the bias should never be 0.

I would also add that using the derivative as it is used in the standard form here isn't quite correct. It would need some filtering. The reason is the change in temp of the drives happens in a step-like fashion, i.e. you'll see a change in temp between two samples and then not see another change for many samples before seeing another change. Personally, I would just leave off the derivative term altogether.

IMHO, this code should be changed to use the velocity form of PI control. It doesn't have the integral windup problem that the standard form has, which you can run into if your fans end up at minimum or maximum. Also IMHO, it is easier to tune.

Code:

dc_change = Kc * ((1 + T/Ti) * err - last_err)

Where Kc = controller gain, T = sample time, and Ti = integral time. Kc and Ti are your tuning variables.

This is the version I use to control the fans on my poweredge, and it works great.

 

[JohnnyGrey]

So, I have a question that might have an obvious answer.

How do I configure the interval when running spincheck.sh if I'm starting it via a postinit script and letting it run? Is there an easier method to starting this script and letting it run? Is there a variable I can change within the script that I overlooked?

(I was testing this out two years ago, but stopped working on it. Now I'm picking it back up, and starting from scratch.

 

[Glorious1]

So, I have a question that might have an obvious answer.

How do I configure the interval when running spincheck.sh if I'm starting it via a postinit script and letting it run? Is there an easier method to starting this script and letting it run? Is there a variable I can change within the script that I overlooked?

(I was testing this out two years ago, but stopped working on it. Now I'm picking it back up, and starting from scratch.

Assuming you really mean spincheck.sh (which just reads and reports data, doesn't control fans):

I can't work with server at this time to check, but reviewing the code on my laptop (VERSION="2018-01-01"), I can't imagine it runs at all. T is number of minutes between cycles. It is never defined in the copy I have. What happens when you run it? Try adding a T=<number of minutes> near the front of the script. That is theoretically the way you would specify it before running.

Sorry, I must have butchered the script at some point. Will correct in due course

 

[JohnnyGrey]

Also, any idea why the stock Intel cooler for an i3 6100T would spin faster at 90% duty cycle than it does at 100%? I'm also pretty sure it's capable of spinning faster than 1700 rpm.

FANA = stock Intel cooler
FAN1 = 6 Corsair ML120's via a Swiftech 8-Way PWM Splitter

Code:

                                ___Duty%___  Curr_RPM____________________
                        MODE     Zone0 Zone1  FANA  FAN1  FAN2  FAN3  FAN4
Duty cycle 100%         Full       100   100  1400  2300   ---   ---   ---
Duty cycle 90%          Full        90    90  1700  2100   ---   ---   ---
Duty cycle 80%          Full        80    80  1600  1900   ---   ---   ---
Duty cycle 70%          Full        70    70  1300  1700   ---   ---   ---
Duty cycle 60%          Full        60    60  1100  1400   ---   ---   ---
Duty cycle 50%          Full        50    50   800  1200   ---   ---   ---
Duty cycle 40%          Full        40    40   900   900   ---   ---   ---
Duty cycle 30%          Full        30    30   900   500   ---   ---   ---
Duty cycle 20%          Full        20    20  1000   400   ---   ---   --- 

When manually changing the Fan Speed Mode to "Full Speed" via the IPMI interface, I see a reported speed of 2000 rpm. Any idea why the script isn't able to max out the CPU speed?

Code:

FAN1             | 2300.000   | RPM        | ok    | 100.000   | 200.000   | 300.000   | 25300.000 | 25400.000 | 25500.000
FAN2             | na         |            | na    | na        | na        | na        | na        | na        | na
FAN3             | na         |            | na    | na        | na        | na        | na        | na        | na
FAN4             | na         |            | na    | na        | na        | na        | na        | na        | na
FANA             | 2000.000   | RPM        | ok    | 300.000   | 500.000   | 700.000   | 25300.000 | 25400.000 | 25500.000

EDIT: Missed a line when copying sensor data

 

[JohnnyGrey]

Assuming you really mean spincheck.sh (which just reads and reports data, doesn't control fans):

I can't work with server at this time to check, but reviewing the code on my laptop (VERSION="2018-01-01"), I can't imagine it runs at all. T is number of minutes between cycles. It is never defined in the copy I have. What happens when you run it? Try adding a T=<number of minutes> near the front of the script. That is theoretically the way you would specify it before running.

Sorry, I must have butchered the script at some point. Will correct in due course

The script works fine. Sorry If I didn't clarify myself. I'm just wondering the proper way of executing the script so it runs for "roughly a day" as you suggest. Do i start it in a terminal window (I'm using putty) and just leave that window up for a day?

 

[zvans18]

Do i start it in a terminal window (I'm using putty) and just leave that window up for a day?

essentially, but you should use something like tmux so it will keep running if the putty connection gets interrupted

 

[JohnnyGrey]

essentially, but you should use something like tmux so it will keep running if the putty connection gets interrupted

I appreciate the assistance. It's now 7:30am and I have about 9 hours worth of readings @ 5 minute intervals. Temps don't fluctuate that much, so I can probably move forward with the configurations. I have three days until my next scheduled scrub, and that's when the temperatures climb above 40°C, so I'd like to get this implemented soon.

 

[JohnnyGrey]

The script is running fine, but is there a way to run it as a postinit script without it occupying the shell on the actual NAS? For example, this is what I see on the iKVM display. I'd like it if the script runs behind the scenes and it sits at the "main menu," if I ever needed to access it for some reason.

 

[Glorious1]

I don't see why it shouldn't run as a post-init script, just like spin*pid.sh. Have you tried it? Problem is wouldn't that require rebooting the machine to run it?

 

[zvans18]

I actually have the same complaint. The console just becomes a log for spinpid2. If I want to do something like configure networking via IPMI, I've taken to killing the postinit and just running spinpid2 in a tmux session.

EDIT: This actually ends up working well for me in a way since I'm adding pairs of 5400 rpm drives to (eventually replace) my 7200 rpm pool, it allows me to tune the set point without restarting.

 

[JohnnyGrey]

I don't see why it shouldn't run as a post-init script, just like spin*pid.sh. Have you tried it? Problem is wouldn't that require rebooting the machine to run it?

What I'm referring to is spinpid2.sh. I'm done with the test scripts. It does run fine as a post-init script. But like @zvans18 said, because it "never ends," it always occupies the console, therefore the main menu is inaccessible. (Console is the word I was looking for, not shell, in my previous reply).

...I've taken to killing the postinit and just running spinpid2 in a tmux session...

Do you have any quick resources or tutorials on tmux sessions? Do you manually launch it? Too bad we couldn't get this to be a service instead of a script.

 

[Glorious1]

I actually have the same complaint. The console just becomes a log for spinpid2. If I want to do something like configure networking via IPMI, I've taken to killing the postinit and just running spinpid2 in a tmux session.

EDIT: This actually ends up working well for me in a way since I'm adding pairs of 5400 rpm drives to (eventually replace) my 7200 rpm pool, it allows me to tune the set point without restarting.

I don't know what console you guys are talking about. Except for booting and changing boot device or BIOS settings or whatever (when I use the IPMI KVM console), I always interact with FreeNAS either through the web GUI or via SSH session. It doesn't show in either one. It doesn't show in the shell in the web GUI. The only time I ever see it is when I cat the log. Your post-init script dialog is set as type 'Script', right?

 

[zvans18]

Do you have any quick resources or tutorials on tmux sessions? Do you manually launch it? Too bad we couldn't get this to be a service instead of a script.

Manually, yeah, if I notice some of my 7200 rpm drives getting a bit warm or if I want to configure networking (rare, but I've been messing with different configurations for the 4 ports of mixed 1G/10G lately, and you can't exactly do that reliably from the web gui) 99% of my use is just 'tmux' to start the session, plus some screen splitting when I'm burning in drives, and 'tmux attach' to bring it back up if i close the window. This guide helped me https://hackernoon.com/a-gentle-introduction-to-tmux-8d784c404340

I don't know what console you guys are talking about. Except for booting and changing boot device or BIOS settings or whatever (when I use the IPMI KVM console), I always interact with FreeNAS either through the web GUI or via SSH session. It doesn't show in either one. It doesn't show in the shell in the web GUI. The only time I ever see it is when I cat the log. Your post-init script dialog is set as type 'Script', right?

We mean the IPMI one. Once FreeNAS boots, the postinit technically never finishes, so we never see "select option 1-12", just the continuous output of spinpid2's log. I mentioned specifically this can interefere with configuring the network as you can't do that 100% reliably from SSH or web gui. But it's rare to need to interface with that console, so for me at least, it's just a minor annoyance. If I need to change something, I just ctrl+c in the IPMI console and start spinpid2 in a tmux session.

And yes, it is set as 'script'
(FWIW, running 11.2-U4.1)

 

[Arubial1229]

If I need to change something, I just ctrl+c in the IPMI console and start spinpid2 in a tmux session.

And yes, it is set as 'script'
(FWIW, running 11.2-U4.1)

Could you just run the tmux session as a postinit command? Then you could detach from the session without needing to manually kill the script.

 

[Glorious1]

Do you have any quick resources or tutorials on tmux sessions? Do you manually launch it? Too bad we couldn't get this to be a service instead of a script.

The basic minimum of tmux:
New named session - nice to name it so you can come back easily
tmux new -s <SESSION NAME>
e.g.:
tmux new -s spin

Then you start the script.
Then exit the session (it keeps running) with:
^b d

Reenter session
tmux attach -t <NAME>
OR
tmux a -t <NAME>

 

[Glorious1]

We mean the IPMI one. Once FreeNAS boots, the postinit technically never finishes, so we never see "select option 1-12", just the continuous output of spinpid2's log. I mentioned specifically this can interefere with configuring the network as you can't do that 100% reliably from SSH or web gui. But it's rare to need to interface with that console, so for me at least, it's just a minor annoyance. If I need to change something, I just ctrl+c in the IPMI console and start spinpid2 in a tmux session.

OK, I guess I've never had any need to get into options 1-12 so haven't run into that. @Arubial1229 's suggestion might work. Might need a script to start the tmux session, start the spin script, and exit. Never tried something like that. Or write a new spin script that doesn't mess with the console. No idea what that would look like.

 

[JohnnyGrey]

...This guide helped me https://hackernoon.com/a-gentle-introduction-to-tmux-8d784c404340

... it's just a minor annoyance...

Appreciate the resource. Yeah I agree. And I'm on U3. I'm not going to update until my new flash drives and USB 2.0 header adapter come in. Long story short, bought another flash drive to match my original from two years ago, but new one is slightly smaller, so can't mirror. I have a brand new 2-pack on the way.

Could you just run the tmux session as a postinit command? Then you could detach from the session without needing to manually kill the script.

This is probably what I'll do. I have yet to decide on whether it's worth it. Either way, launching the script inside of a tmux session via a postinit script will free up the console, should I ever need it. Do tmux sessions persist everywhere? Can I reattach to the session from the shell, SSH, etc?

OK, I guess I've never had any need to get into options 1-12 so haven't run into that. @Arubial1229 's suggestion might work. Might need a script to start the tmux session, start the spin script, and exit. Never tried something like that. Or write a new spin script that doesn't mess with the console. No idea what that would look like.

Yeah, like @zvans18 said, it's just a minor inconvenience. I doubt most people would ever really need to kill the script to access the console menu, especially if their NAS is already up and running. Even if the script doesn't log to the console, wouldn't it still "occupy" the console? And I have no idea how to script. You guys did all the work, and I'm extremely appreciative of that!

 

[Glorious1]

@JohnnyGrey @zvans18 , I've tested this to get what you want. To send all ouput to the log file and NOT to the console, comment out:

Code:

# exec > >(tee -i $LOG) 2>&1

and replace with this:

Code:

exec &> $LOG

Be aware that this way, if you run it manually, like in a tmux session, you will see nothing. I'll include that code as an option in the next version with an explanation.