Supermicro fan control, IPMIUTIL or FEEIPMI?

[Ericloewe]

I still have BIOS 1.1, but I did update the IPMI firmware to 1.42, but I have discovered that "shutdown" in FreeNAS, whether from the GUI or from the console menu, does shut the machine down (confirmed by the "No Signal" message), but it almost immediately powers up again -- not much good when the UPS initiates a shutdown.

Supermicro Tech Support says that it may be necessary to update the BIOS when the IPMI firmware is updated, so I guess that's what I need to try.

Regular ACPI shutdown initiated by the OS works fine for me.

 

[GrumpyBear]

One of the items in the Change List for BIOS rev. 2.0 is:

'Correct BIOS setup item "Power Button Function" behavior'

but I don't know what the problem was that they were fixing (trying to fix?).

I verified before I left for Florida that I'm running the most current BIOS and IPMI versions. (Versions 2.00 and 1.42 respectively, I beleive)

 

[GrumpyBear]

Cruising through the Data Sheets for the C220-series PCH I discovered that they appear to also be able, through the Serial Periphial Interface (SPI) to be able to read tach data from fans and output PWM to them. I couldn't find much more information but SuperMicro does not appear to use this.

Reading the Nuvoton manual though I have discovered that it is possible to bias the AUX and SYS zones PWM output with the CPU temperature. I'm suspecting that SM does do this as evidenced by their somewhat laisse faire attitude with respect to which header the CPU should be connected to. This makes more sense also as the target chassis for a lot of these systems are rack-mount chassis as opposed to mid or full towers with significantly less airflow restrictions.

There are also offsets that can be added to increase and decrease the effective temperature from a sensor and thus tweak the PWM control. I wouldn't recommend tweaking these for the CPU but they might be useful for the X10SL7 where the SYS thermistor appears to suffer from localized heating from the LSI2308

 

[wasp103]

Sorry for digging this up from 2015. What's the current status of your project(s) here?
I'm currently working my way to a direct IPMB-Access to the BMC, hoping to bypass all the higher-level software in the process.
I'm lookung for the Nuvoton Manual you were mentioning and I was hoping you could still provide me with a copy since I can't find anything more detailed about the wpcm450 BMC so far.

My approach is to tap into the I2C/IPMB interface that my X9SRL-F provides.
There's the sourcecode of Intel's IPMItool which offered some insight. And then there's a Wireshark-capture from an unknown IPMB-Communication that I can use as a reference.
But aside from that, there's not much on the interwebs, that I can quickly implement.
This is pure trial and error and feels more like hacking my way through there, than actually producing sophisticated code...

The PWM-generation works, the PID-Controllers work. All I need is some more insight on this protocol.

Did you guys go down that route at some point? Anyone dug into I2C or even IPMB so far?

 

[GrumpyBear]

Sorry for digging this up from 2015. What's the current status of your project(s) here?
I'm currently working my way to a direct IPMB-Access to the BMC, hoping to bypass all the higher-level software in the process.
I'm lookung for the Nuvoton Manual you were mentioning and I was hoping you could still provide me with a copy since I can't find anything more detailed about the wpcm450 BMC so far.

Nuvoton Manual Available here (courtesy of Google) 400+ pages for your reading pleasure. I have yet to locate manuals for the Winbond BMCs who seem to be the other manufacturer one sees on SuperMicro boards.

My approach is to tap into the I2C/IPMB interface that my X9SRL-F provides.
There's the sourcecode of Intel's IPMItool which offered some insight. And then there's a Wireshark-capture from an unknown IPMB-Communication that I can use as a reference.
But aside from that, there's not much on the interwebs, that I can quickly implement.
This is pure trial and error and feels more like hacking my way through there, than actually producing sophisticated code...

The PWM-generation works, the PID-Controllers work. All I need is some more insight on this protocol.

Did you guys go down that route at some point? Anyone dug into I2C or even IPMB so far?

I haven't done any more work on this as looking at it further the risk of really expensive unintended consequences seemed to outweigh my pocketbook. Also for what I want to do with this I realized that using an Arduino, which has PWM built-in, sort-of, with a thermistor or thermal diode glued to one of the disks to implement a PID to control the chassis fans might be easier.

 

[wasp103]

Thanks for the link, although it seems unreachable here...
Nope, not a chance :/ Is TheBigFirewall hitting Germany now?

I'm on the Arduino train already. Had to tinker a bit with the Timer-registers in order to get proper PWM-Output of 25kHz for the fans, but now it works like a charm.

The thing is that those Noctua PPC 3000 PWMs can go as low as 200 rpm and supermicro only supports a 35% duty-cycle as a minimum. But yeah, you prolly know all that already, considering your other post.

 

[Ericloewe]

Thanks for the link, although it seems unreachable here...
Nope, not a chance :/ Is TheBigFirewall hitting Germany now?

Same here, odd...

I'm on the Arduino train already. Had to tinker a bit with the Timer-registers in order to get proper PWM-Output of 25kHz for the fans, but now it works like a charm.

I'm actually planning a fan controller along those lines. I thought about using the ATMega 328P's internal timers, but then I found out about a little I2C/SMBus chip, the Microchip EMC2305. It handles five fans, both PWM output and tachometer readings - the only problem is that it runs off 3.3V, not 5V, though the IO is somewhat 5V-tolerant.

The thing is that those Noctua PPC 3000 PWMs can go as low as 200 rpm

They can? That's way beyond their spec, which is 600 or so.

 

[wasp103]

Same here, odd...


I'm actually planning a fan controller along those lines. I thought about using the ATMega 328P's internal timers, but then I found out about a little I2C/SMBus chip, the Microchip EMC2305. It handles five fans, both PWM output and tachometer readings - the only problem is that it runs off 3.3V, not 5V, though the IO is somewhat 5V-tolerant.

Interesting. I2C is 5V tolerant as well... The thing is, do you know how to implement a firmware for that bugger?

They can? That's way beyond their spec, which is 600 or so.

Mine can, confirmed that while fiddling with the PWM output of my Arduino. Also, his fans can as well:
http://thermalbench.com/2015/03/04/noctua-nf-f12-industrialppc-3000-pwm-120mm-fan/3/
http://thermalbench.com/2015/08/21/noctua-nf-a14-ippc-3000-pwm-140mm-fan/3/

That's my main reason why i bought the 3k over the 2k fans. Maybe that's subject to change in further revisions though...

 

[GrumpyBear]

Thanks for the link, although it seems unreachable here...
Nope, not a chance :/ Is TheBigFirewall hitting Germany now?

Whoops! - it's actually ion a website in Germany so I doubt it's a firewall - more likely me using a VM for Windoze in Linux and clicking the wrong paste button. - it should (might) work now.

 

[GrumpyBear]

Same here, odd...

I really hope you couldn't get to that URL as it means I'm not doing my job very well

I'm actually planning a fan controller along those lines. I thought about using the ATMega 328P's internal timers, but then I found out about a little I2C/SMBus chip, the Microchip EMC2305. It handles five fans, both PWM output and tachometer readings - the only problem is that it runs off 3.3V, not 5V, though the IO is somewhat 5V-tolerant.

Cool - How about a new thread for Aruduino Fan Control Hacking? I've been avoiding getting into to this as it is a rabbit hole I should avoid with all my other activities.

They can? That's way beyond their spec, which is 600 or so.

They likely can. The 140mm one is rated at 800RPM minimum +/-20% so 640-860RPM but that is assuming no back-pressure. I see 600RPM normally and had to set my low critical to under 400 if memory servers me. The PWM Spec from formfactors.org states that PWM duty cycles below 30-35% are undefined but do say that they can be used but a 30-35% Duty-Cycle is required for a couple of seconds first to overcome inertia (stall). Some manufacturers even show 10% on their curves.

 

[wasp103]

Edit: yeah, what GrumpyBear said, basically.
When provided with a tiny PWM-signal, they usually start up to a higher speed (which is clearly audible) and then go down to the lower fan speed to ensure proper startup.

@the Arduino control: Maybe it's about time :)

I really hope you couldn't get to that URL as it means I'm not doing my job very well

No worries, there's no HTTP response. - So no 403 either ;)

 

[Ericloewe]

The thing is, do you know how to implement a firmware for that bugger?

It doesn't need (user-supplied) firmware, you just communicate with it via I2C/SMBus. That should be easy enough, leaving plenty of CPU time available for a serial console interface and some interesting control algorithms, even on a 328P.
When I have the time, I'll have to research microcontrollers more carefully, since I feel that the 328P gets way too expensive for a real product once you add an FTDI USB/RS232 bridge chip (never mind the full-blown microcontroller the Arduino Uno uses...).

 

[wasp103]

The 328p is actually the IC on the Arduino Nano V3 board, the one I use. They go by 5$ each as a test-bed, which IMHO is not too expensive for my tinkering needs...
But what do you have in mind when you talk about "a real product"?

Edit: I think I don't get the whole picture here... In order to control your EMC2305, you would need to use a Microcontroller with "two-wire" communication anyway. Or you buy one of those 250$ USB-I2C-Scticks that are around to do that for you...
But the 328p can do all that for you. the PWM output, the I2C communication, the serial/USB console. and from my testing, handing the PWM work off to the timers, frees the controller to do a lot of other stuff (logging, PID calculations, communicating with some sort of "host", or doing I2C/IPMB Communication directly with the Mainboard's BMC chip)

But on the other hand, maybe we should stop drifting this thread and instead revert to some PMs before actually opening a new Thread. What do you think?

 

[Bidule0hm]

There's clone of the arduino uno on ebay for $3.5 not that expensive... :D

Personally I'd use the serial port of the server, simpler than I2C (I2C is crappy anyway...) and usable with MB who don't have I2C com built-in ;)

 

[Ericloewe]

The 328p is actually the IC on the Arduino Nano V3 board, the one I use. They go by 5$ each as a test-bed, which IMHO is not too expensive for my tinkering needs...
But what do you have in mind when you talk about "a real product"?

Edit: I think I don't get the whole picture here... In order to control your EMC2305, you would need to use a Microcontroller with "two-wire" communication anyway. Or you buy one of those 250$ USB-I2C-Scticks that are around to do that for you...
But the 328p can do all that for you. the PWM output, the I2C communication, the serial/USB console. and from my testing, handing the PWM work off to the timers, frees the controller to do a lot of other stuff (logging, PID calculations, communicating with some sort of "host", or doing I2C/IPMB Communication directly with the Mainboard's BMC chip)

But on the other hand, maybe we should stop drifting this thread and instead revert to some PMs before actually opening a new Thread. What do you think?

There's clone of the arduino uno on ebay for $3.5 not that expensive... :D

Personally I'd use the serial port of the server, simpler than I2C (I2C is crappy anyway...) and usable with MB who don't have I2C com built-in ;)

Well, the architecture that makes sense to me is:

• Server communicates with microcontroller via RS232 (or USB, probably emulating a serial port, anyway)
• Microcontroller handles the control algorithm and tells the EMC2305 what PWM duty cycle to set, via I2C

Interacting with the BMC feels like asking for trouble, since it's intertwined with the system firmware and ME.

The "limiting" factor in a microcontroller-only solution is the pulse counting from the fans. It's not a serious limitation even on a 328P, but it's a significant workload that needs to be planned around.

As for the product, I was thinking of trying to sell a proper PWM fan controller - the only similar thing on the market that an end-user has any chance of using is Corsair's Link system and line of products - but God, that software is atrocious. Truly horrible software.

 

[hotdog]

That piece of **** will not let me connect. I flashed IPMI firmware to latest version (just in case) and also tried to log in via ADMIN account ... still no luck, still the same error about invalid username.

Anyway after cyberjock's fix that "ipmi-sensors-config" is now working from the jail. Well, that command itself is working, but uploading new values for FAN RPM is not possible.

When i change any FAN RPM value to anything else, upload of the config will fail with a error.

Default values:


Setting new value will throw an error:

Even if i try to set some higher value, like 400, it will throw the same error.

But for example CPU temperatures i can change without problems.

Thanks a lot, anyway i am stuck on the first step. It is throwing on me the following message :/



I am 100000000000% sure that the IP and credentials are correct since they are working via web interface and IPMI tool.

Isn't there some settings in bios or something blocking this?

a bit late, but FYI:

I just struggled with the same problem.
solution:
I had to modify the password or add another user in the web gui with a simplier password without an underscore.
Probably other special characters aren't allowed neither...