PID fan controller Perl script

Joined
Dec 2, 2015
Messages
730
This is a problem area with which I have no experience. Google suggests that KCS is one of the potential interfaces to IPMI. Maybe it is the default. Maybe there are other available interfaces that could be configured. Maybe man ipmitool will provide some clues on what other interfaces might be available and how to use them.
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
I'm looking into that... more to learn... :)

As for the new fan script, it appears to be working well. Ramps up and down as needed. However, I noticed that HD temp readings in freenas dashboard didn't decrease but the rpms lowered to 400.
 
Joined
Dec 2, 2015
Messages
730
I'm looking into that... more to learn... :)

As for the new fan script, it appears to be working well. Ramps up and down as needed. However, I noticed that HD temp readings in freenas dashboard didn't decrease but the rpms lowered to 400.
The FreeNAS dashboard is not a great place to monitor HD temps. I created an alias so that if I type "hdt", it runs the following command for n in {0..7}; do (echo -n "da"; echo -n "$n "; smartctl -a -n standby /dev/da$n | grep "Temperature_Celsius" | grep -o "..$");done. This prints out the temperatures of HDs da0 through da7. Or, I check the tail end of the log file tail PID_fan_control.log. Or, if I want to look further back at the last 100 entries, tail -n 100 PID_fan_control.log, or if I want to monitor things going forward, tail -f PID_fan_control.log
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
What's going on with da4/da5? Interesting to me since they are the only HGST drives in the bunch. Others are WD.

I also have ada drives. I tried this, but didn't work:

for n in {0..7}; do (echo -n "ada"; echo -n "$n "; smartctl -a -n standby /dev/ada$n | grep "Temperature_Celsius" | grep -o "..$");done

da0 32
da1 34
da2 33
da3 34
da4 3)
da5 3)
da6 32
da7 32

Also not sure how to read this output:
Code:
root@freenas[~]# tail PID_fan_control.log
05:06:15   33   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   36  -8.00  -0.00    0.00   26.00%
05:07:46   33   35   33   35   40   40   33   32   29   29   31   31   31   31  ^40  37.50 -0.50  Full   466  26/100  34  -8.00  -0.00    0.00  100.00%
05:09:15   33   35   33   35   40   40   33   32   29   28   31   31   31   31  ^40  37.50 -0.50  Full   500 100/100  35  -8.00  -0.00    0.00  100.00%
05:10:45   33   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  37.00 -1.00  Full   366 100/76   33  -8.00  -0.00  -16.00   76.00%
05:12:15   32   35   33   35   39   39   33   32   28   27   30   31   30   30  ^39  37.00 -1.00  Full   333  76/68   33  -8.00  -0.00    0.00   68.00%
05:13:45   32   34   33   34   39   39   33   32   28   27   30   30   30   30  ^39  36.50 -1.50  Full   200  68/40   36 -12.00  -0.00  -16.00   40.00%
05:15:15   32   34   33   34   38   39   32   32   28   27   30   30   30   30  ^39  36.25 -1.75  Full   133  40/26   34 -14.00  -0.00   -8.00   26.00%
05:16:46   32   34   33   34   38   39   32   32   28   27   30   30   30   30  ^39  36.25 -1.75  Full   133  26/26   33 -14.00  -0.00    0.00   26.00%
05:18:16   32   34   33   34   39   39   32   32   28   28   30   31   30   30  ^39  36.50 -1.50  Full   133  26/26   35 -12.00  -0.00    8.00   26.00%
05:19:46   32   34   33   34   39   39   32   32   28   28   31   31   30   30  ^39  36.50 -1.50  Full   133  26/26   35 -12.00  -0.00    0.00   26.00%
 
Last edited:
Joined
Dec 2, 2015
Messages
730
What's going on with da4/da5? Interesting to me since they are the only HGST drives in the bunch. Others are WD.

I also have ada drives. I tried this, but didn't work:

for n in {0..7}; do (echo -n "ada"; echo -n "$n "; smartctl -a -n standby /dev/ada$n | grep "Temperature_Celsius" | grep -o "..$");done

da0 32
da1 34
da2 33
da3 34
da4 3)
da5 3)
da6 32
da7 32
I'm not sure why that is not working. The echo -n "ada"; should print the characters "ada". The | grep -o "..$") portion displays the last two characters from the input line, which are the temperature. It should work, assuming that the command you showed above is exactly the command you used.

Also not sure how to read this output:
Code:
root@freenas[~]# tail PID_fan_control.log
05:06:15   33   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   36  -8.00  -0.00    0.00   26.00%
05:07:46   33   35   33   35   40   40   33   32   29   29   31   31   31   31  ^40  37.50 -0.50  Full   466  26/100  34  -8.00  -0.00    0.00  100.00%
05:09:15   33   35   33   35   40   40   33   32   29   28   31   31   31   31  ^40  37.50 -0.50  Full   500 100/100  35  -8.00  -0.00    0.00  100.00%
05:10:45   33   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  37.00 -1.00  Full   366 100/76   33  -8.00  -0.00  -16.00   76.00%
05:12:15   32   35   33   35   39   39   33   32   28   27   30   31   30   30  ^39  37.00 -1.00  Full   333  76/68   33  -8.00  -0.00    0.00   68.00%
05:13:45   32   34   33   34   39   39   33   32   28   27   30   30   30   30  ^39  36.50 -1.50  Full   200  68/40   36 -12.00  -0.00  -16.00   40.00%
05:15:15   32   34   33   34   38   39   32   32   28   27   30   30   30   30  ^39  36.25 -1.75  Full   133  40/26   34 -14.00  -0.00   -8.00   26.00%
05:16:46   32   34   33   34   38   39   32   32   28   27   30   30   30   30  ^39  36.25 -1.75  Full   133  26/26   33 -14.00  -0.00    0.00   26.00%
05:18:16   32   34   33   34   39   39   32   32   28   28   30   31   30   30  ^39  36.50 -1.50  Full   133  26/26   35 -12.00  -0.00    8.00   26.00%
05:19:46   32   34   33   34   39   39   32   32   28   28   31   31   30   30  ^39  36.50 -1.50  Full   133  26/26   35 -12.00  -0.00    0.00   26.00%
The column headers are displayed every two hours in the log file. If you grab a longer portion of the file you'll see the headers. But, in summary, the first bunch of columns are the individual disk temperatures. Here we see that two of the disks are running much warmer than the others.

The number with a "^" in front is the hottest disk temperature.

The next column is the average temperature of the number of warmest disks that were selected to use.

The next column is the temperature error - we see that generally speaking, the average temperature is lower than the target.

The next column is the IPMI Fan Mode - Full

Next is the fan rpm. I'm puzzled why the reported values are so low. Even at 100% duty cycle we only see 500 rpm reported.

Next is the fan duty cycle before and after the last adjustment.

Next is CPU temperature - 33 to 36 in this case

The next three columns are the outputs of the portions of the fan control that are based on proportional error, integral error and derivative error.

And last we see the fan duty cycle after the last adjustment.

The main point of interest in this bit of log output is that two of the disks are much warmer than the others. It may be worthwhile to review the cooling airflow to attempt to get more air to those two disks. But until then, we see that the loop puts the fans at 100% whenever a disk temperature hits 40. This is to protect the disks even if the user has selected an incorrect set of loop gains. Once all disks are cooler than 40, the control loop kicks in and starts decreasing fan speed to try to get the average temperature up to the target of 38. It never gets there, as the hottest disk will hit 40 first.

For the time being, set the script to control to the average temperature of the warmest two disks - set $hd_num_peak=2 if you are editing directly in the script, or set $config_num_disks = 2 if you are using a config file. To work with two disks, I recommend the following loop gains: Kp=8/3, Ki=0, and Kd=18.
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
Prints it but without values.

Code:
freenas# for n in {0..7}; do (echo -n "ada"; echo -n "$n "; smartctl -a -n standby /dev/ada$n | grep "Temperature_Celsius" | grep -o "..$");done
ada0 9)
1
ada1 0)
0
ada2 0)
ada3 0)
ada4 0)
ada5 0)
ada6 ada7 freenas#
 
Last edited:
Joined
Dec 2, 2015
Messages
730
Try for n in {0..7}; do (echo -n "ada$n "; smartctl -a -n standby /dev/ada$n | grep "Temperature_Celsius");done
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
Code:
freenas# for n in {0..7}; do (echo -n "ada$n "; smartctl -a -n standby /dev/ada$n | grep "Temperature_Celsius");done
ada0 194 Temperature_Celsius     0x0022   072   072   030    Old_age   Always       -       28 (Min/Max 23/29)
231 Temperature_Celsius     0x0033   099   099   005    Pre-fail  Always       -       1
ada1 194 Temperature_Celsius     0x0022   073   073   030    Old_age   Always       -       27 (Min/Max 24/30)
231 Temperature_Celsius     0x0033   100   100   005    Pre-fail  Always       -       0
ada2 194 Temperature_Celsius     0x0022   030   040   000    Old_age   Always       -       30 (0 20 0 0 0)
ada3 194 Temperature_Celsius     0x0022   031   040   000    Old_age   Always       -       31 (0 19 0 0 0)
ada4 194 Temperature_Celsius     0x0022   030   040   000    Old_age   Always       -       30 (0 19 0 0 0)
ada5 194 Temperature_Celsius     0x0022   030   040   000    Old_age   Always       -       30 (0 18 0 0 0)
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
Updated the config file per your feedback. Here's the output:
Code:
freenas# tail -n 50 PID_fan_control.log
08:15:22   32   34   33   34   38   39   33   32   28   27   30   31   30   30  ^39  36.25 -1.75  Full   133  50/26   33 -14.00  -0.00  -16.00   26.00%
08:16:51   32   34   33   34   38   38   33   32   28   28   30   31   30   30  ^38  36.00 -2.00  Full   133  26/26   33 -16.00  -0.00   -8.00   26.00%
08:18:21   32   34   33   34   39   39   33   32   28   28   31   31   31   30  ^39  36.50 -1.50  Full   133  26/30   36 -12.00  -0.00   16.00   30.00%
08:19:51   32   34   33   34   39   39   33   32   29   28   31   31   31   30  ^39  36.50 -1.50  Full   133  30/26   36 -12.00  -0.00    0.00   26.00%
08:21:21   32   34   33   35   39   39   33   32   29   28   31   31   31   30  ^39  36.75 -1.25  Full   133  26/26   35 -10.00  -0.00    8.00   26.00%
08:22:51   32   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   36  -8.00   0.00    8.00   26.00%
08:24:21   33   35   33   35   39   40   33   32   29   29   31   31   31   31  ^40  37.25 -0.75  Full   466  26/100  37  -8.00   0.00    8.00  100.00%
08:25:51   33   35   33   35   40   40   33   32   29   28   31   31   31   31  ^40  37.50 -0.50  Full   500 100/100  35  -8.00   0.00    8.00  100.00%
08:27:21   32   35   33   35   39   39   33   32   28   27   31   31   31   30  ^39  37.00 -1.00  Full   366 100/76   34  -8.00  -0.00  -16.00   76.00%
08:28:51   32   34   33   34   39   39   32   32   28   27   30   31   30   30  ^39  36.50 -1.50  Full   233  76/48   32 -12.00  -0.00  -16.00   48.00%
08:30:21   32   34   33   34   38   39   32   32   28   27   30   30   30   30  ^39  36.25 -1.75  Full   133  48/26   33 -14.00  -0.00   -8.00   26.00%
08:31:51   32   34   33   34   38   39   32   32   28   27   30   30   30   30  ^39  36.25 -1.75  Full   133  26/26   35 -14.00   0.00    0.00   26.00%
08:33:21   32   34   33   34   39   39   32   32   28   28   31   31   30   30  ^39  36.50 -1.50  Full   133  26/26   34 -12.00  -0.00    8.00   26.00%
08:34:51   32   34   33   34   39   39   32   32   28   28   31   31   31   30  ^39  36.50 -1.50  Full   133  26/26   35 -12.00  -0.00    0.00   26.00%
08:36:21   32   34   33   34   39   39   33   32   28   28   31   31   31   30  ^39  36.50 -1.50  Full   133  26/26   35 -12.00  -0.00    0.00   26.00%
08:37:51   32   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   166  26/34   35  -8.00  -0.00   16.00   34.00%
08:39:21   32   35   33   35   40   39   33   32   29   29   31   31   31   31  ^40  37.25 -0.75  Full   466  34/100  35  -8.00  -0.00   16.00  100.00%
08:40:52   32   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   433 100/84   37  -8.00  -0.00   -8.00   84.00%
08:42:21   32   35   33   35   39   39   33   32   29   28   31   31   31   30  ^39  37.00 -1.00  Full   366  84/76   36  -8.00  -0.00    0.00   76.00%
08:43:51   32   35   33   35   39   39   33   32   29   28   31   31   31   30  ^39  37.00 -1.00  Full   333  76/68   33  -8.00  -0.00    0.00   68.00%
08:45:21   32   34   33   34   39   39   33   32   29   28   31   31   31   30  ^39  36.50 -1.50  Full   200  68/40   37 -12.00   0.00  -16.00   40.00%
08:46:51   32   34   33   34   38   38   33   32   29   28   30   31   30   30  ^38  36.00 -2.00  Full   133  40/26   36 -16.00   0.00  -16.00   26.00%
08:48:21   32   34   33   34   39   39   33   32   29   28   31   31   31   30  ^39  36.50 -1.50  Full   133  26/30   38 -12.00  -0.00   16.00   30.00%
08:49:51   32   34   33   34   39   39   33   32   29   29   31   31   31   30  ^39  36.50 -1.50  Full   133  30/26   36 -12.00  -0.00    0.00   26.00%
08:51:21   32   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   166  26/34   37  -8.00   0.00   16.00   34.00%
08:52:51   32   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   133  34/26   36  -8.00  -0.00    0.00   26.00%
08:54:21   32   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   33  -8.00   0.00    0.00   26.00%
08:55:51   33   35   33   35   39   39   33   32   29   29   31   32   31   31  ^39  37.00 -1.00  Full   133  26/26   36  -8.00  -0.00    0.00   26.00%
08:57:21   33   35   33   35   40   40   33   32   29   29   31   32   31   31  ^40  37.50 -0.50  Full   500  26/100  34  -8.00  -0.00    0.00  100.00%
08:58:51   33   36   34   36   40   40   33   32   29   29   31   32   31   31  ^40  38.00  0.00  Full   500 100/100  34  -8.00  -0.00    0.00  100.00%
09:00:21   33   35   34   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   300 100/60   32  -8.00   0.00  -32.00   60.00%
09:01:52   33   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  37.00 -1.00  Full   266  60/52   32  -8.00  -0.00    0.00   52.00%
09:03:22   32   35   33   35   39   39   33   32   28   27   31   31   31   30  ^39  37.00 -1.00  Full   233  52/44   36  -8.00   0.00    0.00   44.00%
09:04:53   32   35   33   35   39   39   33   32   28   27   31   31   31   30  ^39  37.00 -1.00  Full   166  44/36   35  -8.00  -0.00    0.00   36.00%
09:06:22   32   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  37.00 -1.00  Full   133  36/28   35  -8.00   0.00    0.00   28.00%
09:07:52   32   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  37.00 -1.00  Full   133  28/26   35  -8.00  -0.00    0.00   26.00%
09:09:22   32   35   33   35   39   39   33   32   29   28   31   31   31   30  ^39  37.00 -1.00  Full   133  26/26   34  -8.00   0.00    0.00   26.00%
09:10:52   32   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   33  -8.00  -0.00    0.00   26.00%
09:12:22   32   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   39  -8.00   0.00    0.00   26.00%
09:13:52   33   35   33   35   39   40   33   32   29   29   31   31   31   31  ^40  37.25 -0.75  Full   500  26/100  36  -8.00   0.00    0.00  100.00%


PID Fan Controller Log  ---  Target 2 Disk HD Temperature = 38.00 deg C  ---  PID Control Gains: Kp =  2.667, Ki =  0.000, Kd =  18.0
                                                                                 Max   Ave  Temp   Fan   Fan  Fan %   CPU    P      I      D      Fan
2020-01-01 da0  da1  da2  da3  da4  da5  da6  da7 ada0 ada1 ada2 ada3 ada4 ada5 Temp  Temp   Err  Mode   RPM Old/New Temp  Corr   Corr   Corr    Duty
09:15:22   33   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  39.00  1.00  Full   500 100/100  34   4.00  -0.00   21.00  100.00%
09:16:52   33   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  39.00  1.00  Full   500 100/100  34   4.00   0.00    0.00  100.00%
09:18:22   32   35   33   34   39   39   33   32   28   27   30   31   30   30  ^39  39.00  1.00  Full   500 100/100  33   4.00  -0.00    0.00  100.00%
09:19:52   32   34   32   34   38   38   32   32   28   27   30   30   30   30  ^38  38.00  0.00  Full   433 100/88   33   0.00  -0.00  -12.00   88.00%
09:21:22   31   33   32   33   38   38   32   31   27   26   30   30   30   29  ^38  38.00  0.00  Full   433  88/88   33   0.00  -0.00    0.00   88.00%
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
Here we see that two of the disks are running much warmer than the others.

Those two HDs in question are most likely the 7200rpm disks (da4/5), where all the others are 5400.
 
Joined
Dec 2, 2015
Messages
730
Updated the config file per your feedback. Here's the output:
Code:
freenas# tail -n 50 PID_fan_control.log
08:15:22   32   34   33   34   38   39   33   32   28   27   30   31   30   30  ^39  36.25 -1.75  Full   133  50/26   33 -14.00  -0.00  -16.00   26.00%
08:16:51   32   34   33   34   38   38   33   32   28   28   30   31   30   30  ^38  36.00 -2.00  Full   133  26/26   33 -16.00  -0.00   -8.00   26.00%
08:18:21   32   34   33   34   39   39   33   32   28   28   31   31   31   30  ^39  36.50 -1.50  Full   133  26/30   36 -12.00  -0.00   16.00   30.00%
08:19:51   32   34   33   34   39   39   33   32   29   28   31   31   31   30  ^39  36.50 -1.50  Full   133  30/26   36 -12.00  -0.00    0.00   26.00%
08:21:21   32   34   33   35   39   39   33   32   29   28   31   31   31   30  ^39  36.75 -1.25  Full   133  26/26   35 -10.00  -0.00    8.00   26.00%
08:22:51   32   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   36  -8.00   0.00    8.00   26.00%
08:24:21   33   35   33   35   39   40   33   32   29   29   31   31   31   31  ^40  37.25 -0.75  Full   466  26/100  37  -8.00   0.00    8.00  100.00%
08:25:51   33   35   33   35   40   40   33   32   29   28   31   31   31   31  ^40  37.50 -0.50  Full   500 100/100  35  -8.00   0.00    8.00  100.00%
08:27:21   32   35   33   35   39   39   33   32   28   27   31   31   31   30  ^39  37.00 -1.00  Full   366 100/76   34  -8.00  -0.00  -16.00   76.00%
08:28:51   32   34   33   34   39   39   32   32   28   27   30   31   30   30  ^39  36.50 -1.50  Full   233  76/48   32 -12.00  -0.00  -16.00   48.00%
08:30:21   32   34   33   34   38   39   32   32   28   27   30   30   30   30  ^39  36.25 -1.75  Full   133  48/26   33 -14.00  -0.00   -8.00   26.00%
08:31:51   32   34   33   34   38   39   32   32   28   27   30   30   30   30  ^39  36.25 -1.75  Full   133  26/26   35 -14.00   0.00    0.00   26.00%
08:33:21   32   34   33   34   39   39   32   32   28   28   31   31   30   30  ^39  36.50 -1.50  Full   133  26/26   34 -12.00  -0.00    8.00   26.00%
08:34:51   32   34   33   34   39   39   32   32   28   28   31   31   31   30  ^39  36.50 -1.50  Full   133  26/26   35 -12.00  -0.00    0.00   26.00%
08:36:21   32   34   33   34   39   39   33   32   28   28   31   31   31   30  ^39  36.50 -1.50  Full   133  26/26   35 -12.00  -0.00    0.00   26.00%
08:37:51   32   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   166  26/34   35  -8.00  -0.00   16.00   34.00%
08:39:21   32   35   33   35   40   39   33   32   29   29   31   31   31   31  ^40  37.25 -0.75  Full   466  34/100  35  -8.00  -0.00   16.00  100.00%
08:40:52   32   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   433 100/84   37  -8.00  -0.00   -8.00   84.00%
08:42:21   32   35   33   35   39   39   33   32   29   28   31   31   31   30  ^39  37.00 -1.00  Full   366  84/76   36  -8.00  -0.00    0.00   76.00%
08:43:51   32   35   33   35   39   39   33   32   29   28   31   31   31   30  ^39  37.00 -1.00  Full   333  76/68   33  -8.00  -0.00    0.00   68.00%
08:45:21   32   34   33   34   39   39   33   32   29   28   31   31   31   30  ^39  36.50 -1.50  Full   200  68/40   37 -12.00   0.00  -16.00   40.00%
08:46:51   32   34   33   34   38   38   33   32   29   28   30   31   30   30  ^38  36.00 -2.00  Full   133  40/26   36 -16.00   0.00  -16.00   26.00%
08:48:21   32   34   33   34   39   39   33   32   29   28   31   31   31   30  ^39  36.50 -1.50  Full   133  26/30   38 -12.00  -0.00   16.00   30.00%
08:49:51   32   34   33   34   39   39   33   32   29   29   31   31   31   30  ^39  36.50 -1.50  Full   133  30/26   36 -12.00  -0.00    0.00   26.00%
08:51:21   32   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   166  26/34   37  -8.00   0.00   16.00   34.00%
08:52:51   32   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   133  34/26   36  -8.00  -0.00    0.00   26.00%
08:54:21   32   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   33  -8.00   0.00    0.00   26.00%
08:55:51   33   35   33   35   39   39   33   32   29   29   31   32   31   31  ^39  37.00 -1.00  Full   133  26/26   36  -8.00  -0.00    0.00   26.00%
08:57:21   33   35   33   35   40   40   33   32   29   29   31   32   31   31  ^40  37.50 -0.50  Full   500  26/100  34  -8.00  -0.00    0.00  100.00%
08:58:51   33   36   34   36   40   40   33   32   29   29   31   32   31   31  ^40  38.00  0.00  Full   500 100/100  34  -8.00  -0.00    0.00  100.00%
09:00:21   33   35   34   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   300 100/60   32  -8.00   0.00  -32.00   60.00%
09:01:52   33   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  37.00 -1.00  Full   266  60/52   32  -8.00  -0.00    0.00   52.00%
09:03:22   32   35   33   35   39   39   33   32   28   27   31   31   31   30  ^39  37.00 -1.00  Full   233  52/44   36  -8.00   0.00    0.00   44.00%
09:04:53   32   35   33   35   39   39   33   32   28   27   31   31   31   30  ^39  37.00 -1.00  Full   166  44/36   35  -8.00  -0.00    0.00   36.00%
09:06:22   32   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  37.00 -1.00  Full   133  36/28   35  -8.00   0.00    0.00   28.00%
09:07:52   32   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  37.00 -1.00  Full   133  28/26   35  -8.00  -0.00    0.00   26.00%
09:09:22   32   35   33   35   39   39   33   32   29   28   31   31   31   30  ^39  37.00 -1.00  Full   133  26/26   34  -8.00   0.00    0.00   26.00%
09:10:52   32   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   33  -8.00  -0.00    0.00   26.00%
09:12:22   32   35   33   35   39   39   33   32   29   29   31   31   31   31  ^39  37.00 -1.00  Full   133  26/26   39  -8.00   0.00    0.00   26.00%
09:13:52   33   35   33   35   39   40   33   32   29   29   31   31   31   31  ^40  37.25 -0.75  Full   500  26/100  36  -8.00   0.00    0.00  100.00%


PID Fan Controller Log  ---  Target 2 Disk HD Temperature = 38.00 deg C  ---  PID Control Gains: Kp =  2.667, Ki =  0.000, Kd =  18.0
                                                                                 Max   Ave  Temp   Fan   Fan  Fan %   CPU    P      I      D      Fan
2020-01-01 da0  da1  da2  da3  da4  da5  da6  da7 ada0 ada1 ada2 ada3 ada4 ada5 Temp  Temp   Err  Mode   RPM Old/New Temp  Corr   Corr   Corr    Duty
09:15:22   33   35   33   35   39   39   33   32   29   28   31   31   31   31  ^39  39.00  1.00  Full   500 100/100  34   4.00  -0.00   21.00  100.00%
09:16:52   33   35   33   35   39   39   33   32   28   28   31   31   31   30  ^39  39.00  1.00  Full   500 100/100  34   4.00   0.00    0.00  100.00%
09:18:22   32   35   33   34   39   39   33   32   28   27   30   31   30   30  ^39  39.00  1.00  Full   500 100/100  33   4.00  -0.00    0.00  100.00%
09:19:52   32   34   32   34   38   38   32   32   28   27   30   30   30   30  ^38  38.00  0.00  Full   433 100/88   33   0.00  -0.00  -12.00   88.00%
09:21:22   31   33   32   33   38   38   32   31   27   26   30   30   30   29  ^38  38.00  0.00  Full   433  88/88   33   0.00  -0.00    0.00   88.00%
The fan control seems to be working. Right now it is commanding 88% duty cycle, and the two disks of interest are at the target temperature. I suspect that 88% will prove to be a bit faster than needed, and those two disks will eventually cool a bit, allowing the fan to slow down.

If 88% is really needed for steady state, you may find that you don't have enough cooling headroom to handle the disk workload during a scrub. If so, you could switch the HD fans to Noctua NF-F12 industrialPPC 3000 rpm fans - these run twice as fast at the same duty cycle as the ones you have now, giving more cooling capacity when needed. Under normal conditions, the higher speed wouldn't be needed, and the script would end up commanding same fan rpm as now, giving the same noise. But in extreme conditions the fans would be commanded to speed up to cool the HDs.
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
If 88% is really needed for steady state, you may find that you don't have enough cooling headroom to handle the disk workload during a scrub. If so, you could switch the HD fans to Noctua NF-F12 industrialPPC 3000 rpm fans - these run twice as fast at the same duty cycle as the ones you have now, giving more cooling capacity when needed. Under normal conditions, the higher speed wouldn't be needed, and the script would end up commanding same fan rpm as now, giving the same noise. But in extreme conditions the fans would be commanded to speed up to cool the HDs.
Makes sense. I wouldn't want to take any chances. Purchased! Arrives on Friday. When I update the system, I'm assuming all that's needed is to edit $hd_max_fan_speed

Curious now if they will be able to take a lower $hd_fan_duty_low
 
Joined
Dec 2, 2015
Messages
730
Makes sense. I wouldn't want to take any chances. Purchased! Arrives on Friday. When I update the system, I'm assuming all that's needed is to edit $hd_max_fan_speed

Curious now if they will be able to take a lower $hd_fan_duty_low
On my server those fans definitely stall out at 10% duty cycle. They seem to be OK at 12%, but I've set the minimum duty cycle at 16% in my script as I want to have some margin to the limit.
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
Installed new fans, changed hd_fan_duty_low to 20, and max speed to 3000. $fan_duty_low remains at 30.

Code:
freenas# tail -n 100 PID_fan_control.log
17:39:12   30   32   30   32   37   37   30   30   27   26   29   29   29   28  ^37  37.00 -1.00  Full   266  28/26   32  -4.00   0.00    0.00   26.00%
17:40:42   30   32   30   32   37   37   30   30   27   26   29   29   29   29  ^37  37.00 -1.00  Full   266  26/26   35  -4.00  -0.00    0.00   26.00%
17:42:13   30   32   30   32   37   37   30   30   27   26   29   29   29   28  ^37  37.00 -1.00  Full   266  26/26   36  -4.00   0.00    0.00   26.00%
17:43:42   30   32   30   32   37   37   30   30   27   26   29   29   29   29  ^37  37.00 -1.00  Full   266  26/26   34  -4.00  -0.00    0.00   26.00%
17:45:12   30   32   30   32   37   37   30   30   27   26   29   29   29   29  ^37  37.00 -1.00  Full   266  26/26   40  -4.00   0.00    0.00   26.00%
17:46:42   30   32   30   32   37   37   30   30   27   26   29   29   29   29  ^37  37.00 -1.00  Full   266  26/26   32  -4.00  -0.00    0.00   26.00%
17:48:12   30   32   30   32   37   37   30   30   27   26   30   29   29   29  ^37  37.00 -1.00

PID Fan Controller Log  ---  Target 2 Disk HD Temperature = 38.00 deg C  ---  PID Control Gains: Kp =  2.667, Ki =  0.000, Kd =  18.0
                                                                                 Max   Ave  Temp   Fan   Fan  Fan %   CPU    P      I      D      Fan
2020-01-03 da0  da1  da2  da3  da4  da5  da6  da7 ada0 ada1 ada2 ada3 ada4 ada5 Temp  Temp   Err  Mode   RPM Old/New Temp  Corr   Corr   Corr    Duty
17:48:53   30   32   30   32   37   37   30   30   27   26   29   29   29   29  ^37  37.00 -1.00  Full   200  36/20   32  -4.00  -0.00    0.00   20.00%
17:50:23   30   32   30   32   37   37   30   30   27   26   29   29   29   29  ^37  37.00 -1.00  Full   200  20/20   32  -4.00   0.00    0.00   20.00%
17:51:53   30   32   31   32   37   37   30   30   28   26   30   29   29   29  ^37  37.00 -1.00  Full   200  20/20   34  -4.00  -0.00    0.00   20.00%
17:53:23   30   32   31   32   37   37   31   30   28   26   30   29   29   29  ^37  37.00 -1.00  Full   200  20/20   31  -4.00   0.00    0.00   20.00%
17:54:53   31   32   31   32   37   37   31   30   28   27   30   29   29   29  ^37  37.00 -1.00  Full   200  20/20   37  -4.00  -0.00    0.00   20.00%
17:56:24   31   32   31   32   37   37   31   30   28   27   30   29   29   29  ^37  37.00 -1.00  Full   200  20/20   35  -4.00   0.00    0.00   20.00%
17:57:53   31   32   31   32   37   37   31   30   28   27   30   30   29   29  ^37  37.00 -1.00  Full   200  20/20   33  -4.00  -0.00    0.00   20.00%
17:59:23   31   33   31   32   37   37   31   30   28   27   30   30   30   29  ^37  37.00 -1.00  Full   200  20/20   33  -4.00   0.00    0.00   20.00%


PID Fan Controller Log  ---  Target 2 Disk HD Temperature = 38.00 deg C  ---  PID Control Gains: Kp =  2.667, Ki =  0.000, Kd =  18.0
                                                                                 Max   Ave  Temp   Fan   Fan  Fan %   CPU    P      I      D      Fan
2020-01-03 da0  da1  da2  da3  da4  da5  da6  da7 ada0 ada1 ada2 ada3 ada4 ada5 Temp  Temp   Err  Mode   RPM Old/New Temp  Corr   Corr   Corr    Duty
18:00:53   31   33   31   33   38   38   31   30   28   27   30   30   30   30  ^38  38.00  0.00  Full   333  20/32   35   0.00   0.00   12.00   32.00%
18:02:23   31   33   31   33   38   38   31   30   28   27   30   30   30   30  ^38  38.00  0.00  Full   333  32/32   35   0.00   0.00    0.00   32.00%
18:03:53   31   33   31   33   38   38   31   30   28   28   30   30   30   30  ^38  38.00  0.00  Full   333  32/32   36   0.00   0.00    0.00   32.00%
18:05:24   31   33   31   33   38   38   31   30   28   28   30   30   30   30  ^38  38.00  0.00  Full   333  32/32   35   0.00   0.00    0.00   32.00%
18:06:53   31   33   31   33   38   38   31   30   28   28   30   30   30   30  ^38  38.00  0.00  Full   333  32/32   34   0.00   0.00    0.00   32.00%
18:08:24   31   33   31   33   37   37   31   30   28   28   30   30   30   30  ^37  37.00 -1.00  Full   200  32/20   36  -4.00  -0.00  -12.00   20.00%
18:09:53   31   33   31   33   37   37   31   30   28   28   30   30   30   30  ^37  37.00 -1.00  Full   200  20/20   37  -4.00   0.00    0.00   20.00%
18:11:23   31   33   31   33   37   37   31   31   28   28   30   30   30   30  ^37  37.00 -1.00  Full   200  20/20   36  -4.00  -0.00    0.00   20.00%
18:12:53   31   33   32   33   38   38   31   31   28   28   30   30   30   30  ^38  38.00  0.00  Full   333  20/32   32   0.00  -0.00   12.00   32.00%
18:14:23   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   333  32/32   35   0.00  -0.00    0.00   32.00%
18:15:53   31   33   32   33   37   38   31   31   28   27   30   30   30   30  ^38  37.50 -0.50  Full   233  32/24   38  -2.00  -0.00   -6.00   24.00%
18:17:23   31   33   32   33   37   38   31   31   28   27   30   30   30   30  ^38  37.50 -0.50  Full   200  24/22   35  -2.00   0.00    0.00   22.00%
18:18:53   31   33   32   33   37   37   31   31   28   27   30   30   30   30  ^37  37.00 -1.00  Full   200  22/20   35  -4.00  -0.00   -6.00   20.00%
18:20:24   31   33   32   33   37   37   31   31   28   27   30   30   30   30  ^37  37.00 -1.00  Full   200  20/20   35  -4.00   0.00    0.00   20.00%
18:21:53   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   333  20/32   35   0.00   0.00   12.00   32.00%
18:23:23   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   333  32/32   33   0.00   0.00    0.00   32.00%
18:24:53   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   333  32/32   34   0.00   0.00    0.00   32.00%
18:26:23   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   333  32/32   34   0.00   0.00    0.00   32.00%
18:27:53   31   33   32   33   37   37   31   31   28   26   30   30   30   29  ^37  37.00 -1.00  Full   200  32/20   32  -4.00  -0.00  -12.00   20.00%
18:29:23   31   33   32   33   37   37   31   31   28   26   30   30   30   29  ^37  37.00 -1.00  Full   200  20/20   34  -4.00   0.00    0.00   20.00%
18:30:53   31   33   32   33   37   37   31   31   28   27   30   30   30   29  ^37  37.00 -1.00  Full   200  20/20   32  -4.00  -0.00    0.00   20.00%
18:32:23   31   33   32   33   37   37   31   31   28   27   30   30   30   29  ^37  37.00 -1.00  Full   200  20/20   34  -4.00   0.00    0.00   20.00%
18:33:54   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   333  20/32   34   0.00   0.00   12.00   32.00%
18:35:24   31   33   32   33   38   37   31   31   28   27   30   30   30   29  ^38  37.50 -0.50  Full   233  32/24   35  -2.00  -0.00   -6.00   24.00%
18:36:54   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   300  24/30   34   0.00  -0.00    6.00   30.00%
18:38:24   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   300  30/30   34   0.00  -0.00    0.00   30.00%
18:39:54   31   33   32   33   37   38   31   31   28   26   30   30   30   29  ^38  37.50 -0.50  Full   200  30/22   38  -2.00  -0.00   -6.00   22.00%
18:41:24   31   33   32   33   37   38   31   31   28   26   30   30   30   29  ^38  37.50 -0.50  Full   200  22/20   33  -2.00  -0.00    0.00   20.00%
18:42:55   31   33   32   33   37   38   31   31   28   27   30   30   30   29  ^38  37.50 -0.50  Full   200  20/20   34  -2.00   0.00    0.00   20.00%
18:44:24   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  20/26   37   0.00   0.00    6.00   26.00%
18:45:54   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   36   0.00   0.00    0.00   26.00%
18:47:24   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   33   0.00   0.00    0.00   26.00%
18:48:54   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   33   0.00   0.00    0.00   26.00%
18:50:24   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   34   0.00   0.00    0.00   26.00%
18:51:54   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   37   0.00   0.00    0.00   26.00%
18:53:25   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   33   0.00   0.00    0.00   26.00%
18:54:54   31   33   32   33   38   38   31   31   28   28   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   35   0.00   0.00    0.00   26.00%
18:56:24   31   33   32   33   38   38   31   31   29   28   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   35   0.00   0.00    0.00   26.00%
18:57:54   31   33   32   33   38   38   31   31   29   28   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   35   0.00   0.00    0.00   26.00%
18:59:24   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   33   0.00   0.00    0.00   26.00%
19:00:54   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   35   0.00   0.00    0.00   26.00%
19:02:24   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   36   0.00   0.00    0.00   26.00%
19:03:54   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   36   0.00   0.00    0.00   26.00%
19:05:24   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   36   0.00   0.00    0.00   26.00%
19:06:55   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   33   0.00   0.00    0.00   26.00%
19:08:24   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   35   0.00   0.00    0.00   26.00%
19:09:54   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   34   0.00   0.00    0.00   26.00%
19:11:24   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   36   0.00   0.00    0.00   26.00%
19:12:54   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   34   0.00   0.00    0.00   26.00%
19:14:24   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   33   0.00   0.00    0.00   26.00%
19:15:54   31   33   32   33   38   38   31   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   266  26/26   33   0.00   0.00    0.00   26.00%
19:17:24   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   35   0.00   0.00    0.00   26.00%
19:18:54   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   37   0.00   0.00    0.00   26.00%
19:20:24   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   33   0.00   0.00    0.00   26.00%
19:21:54   31   33   32   33   37   38   31   31   28   27   30   30   30   29  ^38  37.50 -0.50  Full   200  26/20   34  -2.00  -0.00   -6.00   20.00%
19:23:24   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  20/26   34   0.00  -0.00    6.00   26.00%
19:24:55   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   36   0.00  -0.00    0.00   26.00%
19:26:24   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   36   0.00  -0.00    0.00   26.00%
19:27:54   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   33   0.00  -0.00    0.00   26.00%
19:29:24   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   33   0.00  -0.00    0.00   26.00%
19:30:54   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   34   0.00  -0.00    0.00   26.00%
19:32:24   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   35   0.00  -0.00    0.00   26.00%
19:33:54   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%
19:35:24   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   35   0.00  -0.00    0.00   26.00%
19:36:54   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   35   0.00  -0.00    0.00   26.00%
19:38:24   31   33   32   33   37   38   31   31   28   26   30   30   30   29  ^38  37.50 -0.50  Full   200  26/20   35  -2.00  -0.00   -6.00   20.00%
19:39:55   31   33   32   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   266  20/26   32   0.00  -0.00    6.00   26.00%
19:41:25   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   35   0.00  -0.00    0.00   26.00%
19:42:55   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   34   0.00  -0.00    0.00   26.00%
19:44:25   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   33   0.00  -0.00    0.00   26.00%
19:45:55   31   33   32   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   266  26/26   34   0.00  -0.00    0.00   26.00%
19:47:25   31   33   32   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   37   0.00  -0.00    0.00   26.00%
19:48:55   31   33   32   33   38   38   31   31   28   27   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
19:50:25   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   266  26/26   35   0.00  -0.00    0.00   26.00%
19:51:55   31   33   32   33   37   38   31   31   28   27   30   30   29   29  ^38  37.50 -0.50  Full   166  26/20   34  -2.00  -0.00   -6.00   20.00%
19:53:25   31   33   32   33   38   38   31   31   28   27   30   30   30   29  ^38  38.00  0.00  Full   233  20/26   35   0.00  -0.00    6.00   26.00%

 
Joined
Dec 2, 2015
Messages
730
The script seems to be working well under the conditions tested.

Please confirm that the only changes you made were to the variables. You didn't need to make any changes to the IPMI raw commands to get it to work with your X11 board.
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
It will be interesting to see how it responds under load. Currently, the only changes I made were to the config file. I didn't run any ipmi raw commands.

Is "nc" an issue on Fan1? Currently $fan_duty_low is set to 30.
FAN1 | 600 RPM | nc

Code:
PID Fan Controller Log  ---  Target 2 Disk HD Temperature = 38.00 deg C  ---  PID Control Gains: Kp =  2.667, Ki =  0.000, Kd =  18.0
                                                                                 Max   Ave  Temp   Fan   Fan  Fan %   CPU    P      I      D      Fan
2020-01-04 da0  da1  da2  da3  da4  da5  da6  da7 ada0 ada1 ada2 ada3 ada4 ada5 Temp  Temp   Err  Mode   RPM Old/New Temp  Corr   Corr   Corr    Duty
04:01:07   31   33   32   33   38   38   32   31   29   27   31   30   30   30  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
04:02:37   31   33   32   33   38   38   32   31   29   27   31   30   30   30  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
04:04:07   31   33   32   33   38   38   32   31   29   27   31   30   30   30  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:05:37   31   33   32   33   38   38   32   31   29   27   31   30   30   30  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
04:07:07   31   33   32   33   38   38   32   31   28   27   31   30   30   30  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:08:37   31   33   32   33   38   38   32   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   34   0.00  -0.00    0.00   26.00%
04:10:07   31   33   32   33   38   38   32   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:11:37   31   33   32   33   38   38   32   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:13:07   31   33   32   33   38   38   32   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%
04:14:37   31   33   32   33   38   38   32   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   37   0.00  -0.00    0.00   26.00%
04:16:07   31   33   32   33   38   38   32   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
04:17:37   31   33   32   33   38   38   32   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:19:07   31   33   32   33   38   38   32   31   28   27   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:20:37   31   33   32   33   38   38   32   31   28   26   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
04:22:07   31   33   32   33   38   38   32   31   28   26   30   30   30   30  ^38  38.00  0.00  Full   233  26/26   37   0.00  -0.00    0.00   26.00%
04:23:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
04:25:07   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
04:26:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%
04:28:07   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:29:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   34   0.00  -0.00    0.00   26.00%
04:31:07   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:32:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
04:34:07   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:35:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   37   0.00  -0.00    0.00   26.00%
04:37:07   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:38:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   34   0.00  -0.00    0.00   26.00%
04:40:07   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   34   0.00  -0.00    0.00   26.00%
04:41:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:43:07   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:44:38   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   34   0.00  -0.00    0.00   26.00%
04:46:07   31   33   32   33   38   38   32   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%
04:47:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:49:07   31   33   32   33   38   38   32   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:50:37   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:52:07   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:53:38   31   33   32   33   38   38   32   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:55:08   31   33   32   33   38   38   32   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%
04:56:38   31   33   32   33   38   38   32   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
04:58:08   31   33   31   33   38   38   32   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
04:59:39   31   33   31   33   38   38   32   31   28   26   30   29   29   29  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
05:01:09   31   33   31   33   38   38   32   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   35   0.00  -0.00    0.00   26.00%
05:02:39   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
05:04:10   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%
05:05:40   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
05:07:10   31   33   31   33   38   38   31   31   28   26   30   29   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
05:08:40   31   33   31   33   38   38   31   31   28   26   30   29   29   29  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
05:10:10   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
05:11:40   31   33   31   33   37   38   31   31   28   26   30   30   29   29  ^38  37.50 -0.50  Full   166  26/20   33  -2.00  -0.00   -6.00   20.00%
05:13:10   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  20/26   33   0.00  -0.00    6.00   26.00%
05:14:40   31   33   31   33   38   38   31   31   28   26   30   29   29   29  ^38  38.00  0.00  Full   233  26/26   34   0.00  -0.00    0.00   26.00%
05:16:10   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
05:17:40   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
05:19:10   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   33   0.00  -0.00    0.00   26.00%
05:20:40   31   33   31   33   38   38   31   31   28   26   30   30   30   29  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%
05:22:10   31   33   31   33   38   38   31   31   28   26   30   29   29   29  ^38  38.00  0.00  Full   233  26/26   36   0.00  -0.00    0.00   26.00%
05:23:40   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%
05:25:10   31   33   31   33   38   38   31   31   28   26   30   30   29   29  ^38  38.00  0.00  Full   233  26/26   32   0.00  -0.00    0.00   26.00%


Code:
freenas# ipmitool sdr | grep FAN # check fan speeds
FAN1             | 600 RPM           | nc
FAN2             | no reading        | ns
FAN3             | no reading        | ns
FAN4             | 900 RPM           | ok
FANA             | 700 RPM           | ok

Code:
freenas# for n in {0..7}; do (echo -n "ada$n "; smartctl -a -n standby /dev/ada$n | grep "Temperature_Celsius");done
ada0 194 Temperature_Celsius     0x0022   072   072   030    Old_age   Always       -       28 (Min/Max 26/29)
231 Temperature_Celsius     0x0033   099   099   005    Pre-fail  Always       -       1
ada1 194 Temperature_Celsius     0x0022   074   074   030    Old_age   Always       -       26 (Min/Max 25/28)
231 Temperature_Celsius     0x0033   100   100   005    Pre-fail  Always       -       0
ada2 194 Temperature_Celsius     0x0022   030   040   000    Old_age   Always       -       30 (0 20 0 0 0)
ada3 194 Temperature_Celsius     0x0022   029   040   000    Old_age   Always       -       29 (0 19 0 0 0)
ada4 194 Temperature_Celsius     0x0022   029   040   000    Old_age   Always       -       29 (0 19 0 0 0)
ada5 194 Temperature_Celsius     0x0022   029   040   000    Old_age   Always       -       29 (0 18 0 0 0)
ada6 ada7 freenas# for n in {0..7}; do (echo -n "da$n "; smartctl -a -n standby /dev/da$n | grep "Temperature_Celsius");done
da0 194 Temperature_Celsius     0x0022   119   115   000    Old_age   Always       -       31
da1 194 Temperature_Celsius     0x0022   119   113   000    Old_age   Always       -       33
da2 194 Temperature_Celsius     0x0022   119   113   000    Old_age   Always       -       31
da3 194 Temperature_Celsius     0x0022   119   115   000    Old_age   Always       -       33
da4 194 Temperature_Celsius     0x0002   157   157   000    Old_age   Always       -       38 (Min/Max 20/43)
da5 194 Temperature_Celsius     0x0002   157   157   000    Old_age   Always       -       38 (Min/Max 21/43)
da6 194 Temperature_Celsius     0x0022   119   114   000    Old_age   Always       -       31
da7 194 Temperature_Celsius     0x0022   119   116   000    Old_age   Always       -       31
 
Joined
Dec 2, 2015
Messages
730
It will be interesting to see how it responds under load. Currently, the only changes I made were to the config file. I didn't run any ipmi raw commands.

Is "nc" an issue on Fan1? Currently $fan_duty_low is set to 30.
FAN1 | 600 RPM | nc



Code:
freenas# ipmitool sdr | grep FAN # check fan speeds
FAN1             | 600 RPM           | nc
FAN2             | no reading        | ns
FAN3             | no reading        | ns
FAN4             | 900 RPM           | ok
FANA             | 700 RPM           | ok
The ”nc“ indicates that FAN1 speed is less than the lower non-critical threshold. See https://www.ixsystems.com/community...nge-ipmi-sensor-thresholds-using-ipmitool.35/

Options:
  • Lower the thresholds for that fan
  • Increase the minimum duty cycle in the script
  • Ignore it, as the BMC doesn’t bump all fans to 100% until the speed decreases below the lower critical value (or possibly the lower non-recoverable - I’m not sure which)
I vote for the first option.
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
Alrighty! Set it to sensor thresh lower 300 400 500

That impi warm reset was great!

Code:
freenas# ipmitool sensor list all
CPU Temp         | 25.000     | degrees C  | ok    | 0.000     | 0.000     | 0.000     | 95.000    | 100.000   | 100.000
PCH Temp         | 31.000     | degrees C  | ok    | 0.000     | 5.000     | 16.000    | 90.000    | 95.000    | 100.000
System Temp      | 25.000     | degrees C  | ok    | -10.000   | -5.000    | 0.000     | 80.000    | 85.000    | 90.000
Peripheral Temp  | 27.000     | degrees C  | ok    | -14.000   | -9.000    | -4.000    | 76.000    | 81.000    | 86.000
VcpuVRM Temp     | 30.000     | degrees C  | ok    | -5.000    | 0.000     | 5.000     | 95.000    | 100.000   | 105.000
DIMMA1 Temp      | na         |            | na    | na        | na        | na        | na        | na        | na
DIMMA2 Temp      | 26.000     | degrees C  | ok    | -5.000    | 0.000     | 5.000     | 80.000    | 85.000    | 90.000
DIMMB1 Temp      | na         |            | na    | na        | na        | na        | na        | na        | na
DIMMB2 Temp      | 26.000     | degrees C  | ok    | -5.000    | 0.000     | 5.000     | 80.000    | 85.000    | 90.000
FAN1             | 600.000    | RPM        | ok    | 300.000   | 400.000   | 500.000   | 25300.000 | 25400.000 | 25500.000
FAN2             | na         |            | na    | na        | na        | na        | na        | na        | na
FAN3             | na         |            | na    | na        | na        | na        | na        | na        | na
FAN4             | 900.000    | RPM        | ok    | 300.000   | 500.000   | 700.000   | 25300.000 | 25400.000 | 25500.000
FANA             | 500.000    | RPM        | ok    | 100.000   | 200.000   | 300.000   | 25300.000 | 25400.000 | 25500.000
12V              | 12.320     | Volts      | ok    | 10.144    | 10.272    | 10.784    | 12.960    | 13.280    | 13.408
5VCC             | 4.948      | Volts      | ok    | 4.246     | 4.298     | 4.480     | 5.390     | 5.546     | 5.598
3.3VCC           | 3.333      | Volts      | ok    | 2.789     | 2.823     | 2.959     | 3.554     | 3.656     | 3.690
VBAT             | 2.972      | Volts      | ok    | 2.384     | 2.496     | 2.580     | 3.476     | 3.588     | 3.672
VCPU             | 1.192      | Volts      | ok    | 0.076     | 0.076     | 0.076     | 1.516     | 1.516     | 1.516
VDIMMAB          | 1.191      | Volts      | ok    | 0.948     | 0.975     | 1.047     | 1.344     | 1.425     | 1.443
5VSB             | 5.078      | Volts      | ok    | 4.246     | 4.376     | 4.480     | 5.390     | 5.546     | 5.598
3.3VSB           | 3.248      | Volts      | ok    | 2.789     | 2.891     | 2.959     | 3.554     | 3.656     | 3.690
VBMC 1.2V        | 1.209      | Volts      | ok    | 1.020     | 1.047     | 1.092     | 1.344     | 1.371     | 1.398
VPCH 1.0V        | 1.016      | Volts      | ok    | 0.872     | 0.890     | 0.917     | 1.043     | 1.052     | 1.070
Chassis Intru    | 0x0        | discrete   | 0x0000| na        | na        | na        | na        | na        | na


would the fan script cause, what appears to be, writing to the disks every few seconds
 
Last edited:

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
would the fan script cause, what appears to be, writing to the disks every few seconds
The fan script writes to logs (if the variables are set as default) every cycle... by default that cycle is not every few seconds, so unless you "customized" it, no, this script can't be the cause.
 

thepixelgeek

Patron
Joined
Jan 21, 2016
Messages
271
The fan script writes to logs (if the variables are set as default) every cycle... by default that cycle is not every few seconds, so unless you "customized" it, no, this script can't be the cause.
Thanks. I only customized fan speeds.
 
Top