Fan Scripts for Supermicro Boards Using PID Logic

Fan Scripts for Supermicro Boards Using PID Logic 2020-08-20, previous one was missing a file

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
Right, for whatever reason (which apparently @Ericloewe identified - how can one person know so much?) some of those X9s seem to have different raw commands.

I think the fan mode commands are the same, but go into your IPMI gui and see how many fan modes are listed. The fan mode reading/setting may have to be modified slightly. In that modified script, he has set mode manually before running it, so he may have removed the setting part.

The raw commands for duty cycle reading/setting are different, so you can make those changes. It sounds like you are getting those to work?

For CPU, for now try changing "CPU Temp" to "CPU1 Temp". You could read both CPUs, but you would have to modify the output formatting to show it, and believe me that's a PITA. Maybe better would be to read them both, then use the highest one as CPU temperature.
EDIT: Or you could use the sysctl method for CPU temp as in spincheck for the other scripts - it seems to be working for you.

Reading RPMs seems to be working; that's not a raw command. For reading and displaying the additional fans, you get into the fun of formatting output. Just copy the stuff for fans 1-4 to 5-6 in the header and the data line. Then the header lines will lose alignment etc.

No idea about the CPU log - maybe a slight typo in the path or something?

EDIT: This has some good stuff too: https://www.reddit.com/r/homelab/comments/8fhomj/getting_those_supermicro_x9_motherboard_fans/ I don't think the duty needs to be given in the raw command as hex, at least it doesn't for the other boards.
 
Last edited:

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
I don't see how line 310 in the modified script could work:
Code:
      echo -n "$($IPMITOOL raw 0x30 0x91 0x5A 0x3 0x1"$ZONE" "$SPEED")"

The last hex number, 0x1, should represent the zone (I don't know which one has that value on those boards), but zone is given again as a variable.
 

zvans18

Dabbler
Joined
Sep 6, 2016
Messages
23
I don't see how line 310 in the modified script could work:
Code:
      echo -n "$($IPMITOOL raw 0x30 0x91 0x5A 0x3 0x1"$ZONE" "$SPEED")"

The last hex number, 0x1, should represent the zone (I don't know which one has that value on those boards), but zone is given again as a variable.
Quck reply, I think it's determining 0x10 vs 0x11 for zone, then the last one is 0x<0-255 in hex> for speed EDIT: nope, just 0-255 for speed works
 
Last edited:

zvans18

Dabbler
Joined
Sep 6, 2016
Messages
23
Update: Finally had a little bit of time. Ultimately not much more effort than dragging and dropping the different IPMI raw commands over and double checking the 0-100 stuff is corrected to 0-255. This is my current script which seems to fully work (including setting fan mode) with the exception of the first part of function read_fan_data (lines 130-135). Before that, it would throw in a ton of "Unable to send RAW command (channel=0x0 netfn=0x30 lun=0x0 cmd=0x70 rsp=0xcc): Invalid data field in request" and mess up all the formatting. This could probably work if I knew the correct raw command, but in the meantime it seems to not matter. I plugged in a 40mm fan to FANA and left it loose in the case just in case not having one would throw something off. I also excluded HGST drives as those are my hot 7200 rpm drives I have in another enclosure. Next up I could mess around with formatting to show all my fans.

Honestly, my main concern seems to be board temperatures. When smaller fans are slowed down, they just really don't seem to move enough air. After manually setting fans to minimum, I got a CATERR after an hour or so which I'm hoping was due to heat? I'll be living with standard speed until I can make hardware changes. Next step will likely be removing/limiting/changing rear fans, switching the midcase fan wall for a 3x120mm version, and maybe adding a standalone fan closer to PCIe cards and/or 40mm fans on card heatsinks. I'm switching CPUs and adding active cooling so the shroud will be removed anyway.

ANYWAY, thank you again for this wonderful set of scripts, they've really made having servers in the house a much better experience
 

Attachments

  • spinpid2z.sh.txt
    20.5 KB · Views: 779

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
This is my current script which seems to fully work (including setting fan mode) with the exception of the first part of function read_fan_data (lines 130-135). Before that, it would throw in a ton of "Unable to send RAW command (channel=0x0 netfn=0x30 lun=0x0 cmd=0x70 rsp=0xcc): Invalid data field in request" and mess up all the formatting. This could probably work if I knew the correct raw command.
Right, so now you're not reading duty cycle, you're just assuming it is what was set, which is the way the script always worked in the past. Shouldn't be a problem. But I don't know why the manufacturers make it so difficult to get the raw commands.
Honestly, my main concern seems to be board temperatures. When smaller fans are slowed down, they just really don't seem to move enough air. After manually setting fans to minimum, I got a CATERR after an hour or so which I'm hoping was due to heat? I'll be living with standard speed until I can make hardware changes.
If you're using the script, the fans should speed up when things get hot. If you're setting mode to standard, I guess you're not using the script. Don't know what CATERR is.
 

IronRobi

Explorer
Joined
Apr 15, 2016
Messages
52
Saw the modified script for X9DRi-LN4F+ boards posted by @zvans18 so figured I'd finally give this a shot.

Seeing all kinds of "
Unable to send RAW command (channel=0x0 netfn=0x30 lun=0x0 cmd=0x91 rsp=0x82): Unknown (0x82)" in the logs and fan speeds aren't being set.

Also a config question.
On the CPU header, I have my 2 Supermicro active heat syncs on FAN1 and FAN6, I also have 2 Noctua F12 IPPC 3000 directed under the shrowd connected to FAN2 and FAN3.
Then on the Peripheral header I have 3 more Noctua fan's on FANA and FANB for the drives.

My question comes from the CPU settings.
RPM_CPU_30=2550 # Your system
RPM_CPU_MAX=8500

I have them set based off the heatsync fans, but I'm assuming with these settings it will run the 2 Noctua's on that header at almost max pretty much full time?
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
Need to know what board you have, and specifically if it has an Aspeed BMC. Clearly it has dual fan/cooling zones.
 

IronRobi

Explorer
Joined
Apr 15, 2016
Messages
52
Need to know what board you have, and specifically if it has an Aspeed BMC. Clearly it has dual fan/cooling zones.
The board is a Supermicro X9DRi-LN4F+ no clue on the Aspeed BMC.

I'm just loading up the original version of the script now to see if that one works, and I changed my fan config around a bit and moved all the Noctua's onto CPU and the CPU ones onto FANA/B. In my head it will be easier to control that way
 

IronRobi

Explorer
Joined
Apr 15, 2016
Messages
52
Alright, with the original version of the script I'm getting "
Unable to send RAW command (channel=0x0 netfn=0x30 lun=0x0 cmd=0x70 rsp=0xcc): Invalid data field in request"

So instead of coming up with the error being unknown, it's specifying invalid data field which is the same error originally seen by @zvans18
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
Yeah, in your link it shows Nuvoton WPCM450 BMC. I think that’s different from zvans, so probably a third set of raw commands.

EDIT - Wait, that's the same BMC that @zvans18 has in his board. The same commands should work. Were I you, I would experiment with raw commands directly in console or ssh. See if you can set mode to full first.
 
Last edited:

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
Glorious1 updated Fan Scripts for Supermicro Boards Using PID Logic with a new update entry:

Happy Halloween

Changes for version 2019-11-01 since 2019-10-04. All scripts changed a bit. Here are bigger changes:

A. The two fan control scripts are now set up for configuration files, which should go in the same directory as the script. This way it is easier to install new script versions; you won't have to fiddle with settings unless there are new ones.

B. In spinpid2.sh, added a setting to tell the script how to determine what the duty cycles (%) are. The options are assuming they are what the...

Read the rest of this update entry...
 

barbierimc

Dabbler
Joined
Jun 25, 2016
Messages
22
Hi @Glorious1, thanks for the great script.

I'm running FreeNAS as a VM on ESXi and had to make a few small changes to make this work which I thought I'd pass back.

Firstly, I need to make use of the -H host option on ipmitool to remotely execute commands on the host. To make this more flexible and easier to maintain I moved the IPMITOOL= definition before the config file is sourced. This allows for overriding the IPMITOOL definition in the config file.

In an ESXi VM the cpu core temperature isn't exposed to sysctl so I had to revert to the older ipmitool approach. Again, in the interest of flexibility I added a check near the start of the script execution to check for the existence of dev.cpu.0.temperature. Based on the result, the most suitable method for getting the cpu temperature is used.

I also needed to add '/VMware/d' in the DEVLIST filter to exclude VMware virtual disks.

Finally, I added two function hooks at the end of the CPU_check_adjust() and DRIVES_check_adjust() routines. This allows two 'POST' functions to be optionally included in the config file that are executed at the end of each check cycle. In my case I use them to push values such as Tmean, Tmax, SP, DUTY, CPU_Temp, CPU_Ref, etc. into an influx db for graphing in Grafana. This was really useful for tuning and monitoring rather than using the log files.

I've attached a copy with the changes in case you want to pull them into your script as they may be useful for others.

Thanks again
 

Attachments

  • spinpid2.sh.txt
    20.5 KB · Views: 575
  • spinpid2.config.txt
    4.6 KB · Views: 510
Last edited:

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
Hi @Glorious1, thanks for the great script.
I'm running FreeNAS as a VM on ESXi and had to make a few small changes to make this work which I thought I'd pass back.
Wow, you're way ahead of me man - I don't even know what VM on ESXi is. But I'll try to use those changes next time I update it. I'll probably just put the IPMITOOL definition in the config file rather than define it in both places. I don't think the CPU temp is necessarily any better one method or the other. I like to keep it simple so maybe will just use the ipmitool approach by default and leave the alternative commented out in case someone prefers to do it the fancy way. I never heard of Grafana but have to imagine that's not going to be widely used, but I'll leave those function declarations in there commented out in case anyone wants to do that.

Could you send me your config file just so I'm sure I understand what you're doing?
 

barbierimc

Dabbler
Joined
Jun 25, 2016
Messages
22
Wow, you're way ahead of me man - I don't even know what VM on ESXi is. But I'll try to use those changes next time I update it. I'll probably just put the IPMITOOL definition in the config file rather than define it in both places. I don't think the CPU temp is necessarily any better one method or the other. I like to keep it simple so maybe will just use the ipmitool approach by default and leave the alternative commented out in case someone prefers to do it the fancy way. I never heard of Grafana but have to imagine that's not going to be widely used, but I'll leave those function declarations in there commented out in case anyone wants to do that.

Could you send me your config file just so I'm sure I understand what you're doing?

Sure, I'll post the config file tonight. The main reason I left the IPMITOOL definition in the script and moved it ahead of the config source is so that the script wouldn't break if anyone upgraded and didn't add the definition to their existing config. You could put a commented sample override statement in the config as a guide, but leave the definition in the main script, just earlier in the execution as I have done.

Same with the way I've made the other changes. It should still function correctly without any existing changes to your standard config. I didn't want to make a breaking change. For comments sake, you could put empty POST_CPU_check_adjust() functions in your default config

Grafana with InfluxDB is pretty cool, and a great way to capture and chart system data https://grafana.com/grafana/dashboards/2162
 

StarkJohan

Explorer
Joined
Mar 27, 2015
Messages
62
I've been running this excellent script on freenas bare metal for years but recently moved my freenas install to a VM on Proxmox. I went a different route than barbierimc as I decided to run the script on the proxmox host. Proxmox is debian so I modified to use lm-sensors (and lsblk for the disks on one of my pve nodes not running freenas) but all the hdds are passed through to freenas via the HBA so I fetched the disk info (camcontrol as well as smartctl) over ssh.

Thanks for a very nice and flexible script! If anyone is in a similar situation I'd be happy to supply my dirty modded scripts.
 

msbxa

Contributor
Joined
Sep 21, 2014
Messages
151
Looking for Fan scripts for Supermicro X9SRL-F with Nuvoton WPCM450 BMC
 

xontik

Cadet
Joined
May 26, 2020
Messages
1
Hi, I modified this script to work with X9Dri-f inside an ubuntu vm :
- now using all fan header pins
- get the hotter cpu of both
- override drive fan with cpu fan if need more cooling (under heavy load)

Feel free to point out mistakes

 

wah00kid

Dabbler
Joined
Nov 24, 2015
Messages
37
Looks great thanks for the wonderful tool.

I am getting this error when running the latest script versions of the spincheck.sh:


Code:
Wednesday, May 27
          da0  da1  da2  da3  ada1 ada3 Tmax Tmean  ERRc CPU  FAN1  FAN2  FAN3  FAN4  FANA Fan%0 Fan%1 MODE
21:53:29  *40  *43  *34  ./spincheck.sh: line 120: let: Tsum += : syntax error: operand expected (error token is "+= ")
*0   *34  *36  ^43  31.17 -2.40  46  1000   800  1500  1400  1900     0     0 Optimal^C


When looking at my devices it is faulting at da3 which is a SAS drive.

I only have two SAS drives in the machine so I suppose I could take them out of the list but if there is an easy fix I am down for that too.

Thanks
 

Glorious1

Guru
Joined
Nov 23, 2014
Messages
1,211
Looks great thanks for the wonderful tool.

I am getting this error when running the latest script versions of the spincheck.sh:


Code:
Wednesday, May 27
          da0  da1  da2  da3  ada1 ada3 Tmax Tmean  ERRc CPU  FAN1  FAN2  FAN3  FAN4  FANA Fan%0 Fan%1 MODE
21:53:29  *40  *43  *34  ./spincheck.sh: line 120: let: Tsum += : syntax error: operand expected (error token is "+= ")
*0   *34  *36  ^43  31.17 -2.40  46  1000   800  1500  1400  1900     0     0 Optimal^C


When looking at my devices it is faulting at da3 which is a SAS drive.

I only have two SAS drives in the machine so I suppose I could take them out of the list but if there is an easy fix I am down for that too.

Thanks
Could you post the output of sudo smartctl -a -n standby /dev/da3?
I don't know anything about SAS drives but could probably modify if I know how they respond.
 
Top