Out of my league: How do I adjust fan speed (Supermicro CSE-846, X9DRi-F) - fans quiet with hot disks

esamett

Patron
Joined
May 28, 2011
Messages
345
lists different numbers to try... will report back:

ipmitool raw 0x30 0x91 0x5A 0x3 0x11 0x7f sets FANA-B to half speed.
ipmitool raw 0x30 0x91 0x5A 0x3 0x10 0x7f sets FAN1-4 to half speed.
ipmitool raw 0x30 0x91 0x5A 0x3 0x10 0xff sets FAN1-4 to full speed.
 

esamett

Patron
Joined
May 28, 2011
Messages
345
I found the Rosetta Stone of fan speed commands for my X9 motherboard:

the basic command for "half speed" is: ipmitool raw 0x30 0x91 0x5A 0x3 0x10 0x7f which works out to about 4300 RPM for system fans. The last two digits are hexadecimal for fan speed, 0-256.

I started a disk scrub on both arrays to stress the system and then cycled through the disks and found the hottest:
smartctl -a /dev/da13 | grep -i temperature.

When temps exceeded 34' I increased the fan speed (table below) to af (175) and left overnight. Disk temp this morning was 32-33'C with scrub still running. Noise is tolerable. The server is in the basement and is not noticeable with door closed.

I presume that the fan speed will reset to Max when I reboot so my next task will be to figure out how to make the fan command part of the bootup script:
ipmitool raw 0x30 0x91 0x5A 0x3 0x10 0xaf

-------------------------------------
hex.dec.fan speed.ratio
8f 143 5000/143=35
9f 159 5175/159=32.5
af 175 5700/175=32.6 overnight disk temp 32-33'C
bf 191
cf 207
df 223
ef 239
ff 255 7000/255=27.5
 

esamett

Patron
Joined
May 28, 2011
Messages
345

Fail: I entered the ipmitool command as a preinit startup command and rebooted. Fans running full speed.


Success: Running IPMITOOL as a startup command requires the full path. This is found with:
[root@freenas ~]# which ipmitool
/usr/local/bin/ipmitool
With information above I modified command to include the full path to command. Rebooted to proper fan speed.
/usr/local/bin/ipmitool raw 0x30 0x91 0x5A 0x3 0x10 0xaf

Thanks to all for your help. If I get inspired I can try to use the speed sensing scripts.

would get my script journey started I think.

For now i am satisfied with the relative quiet and coolness of my "simple" solution.
 

mikedeissler

Dabbler
Joined
Aug 26, 2011
Messages
19
Please excuse my inexperience. This is my first excursion into using server hardware. I just "finished" moving my FreeNAS from consumer hardware to the server case. 2x11-Z3 disk arrays, 2 spares. dual SSD boot drives inside case powered from extra Molex connector from power distributor. Set up network NIC.

When I turn on system the 5 big case fans spin up loudly initially and then get rather quiet when Motherboard boots. I promptly got disk overheat errors from FreeNAS, even when I replaced the case cover to promote air flow. I rebooted several times and got similar disk overtemperature warnings. Since the system fans currently spin slowly and quietly I want to adjust the speed to increase cooling. Its my guess that the fans can cool my disks when spun up to speed. I looked through the BIOS menus and found no fan control options. I search the forum and found discussions about "scripts" and "IPMI" of which I have no knowledge.

Please guide me in the right direction about fan control. I would hate to have to go back to my consumer frankenstein build.

I had this problem and it ended up being the CPU cooler. TrueNAS monitors the sensors on your MB (fanspeed, temp etc.) you can see what temp TN is getting by typing :

sysctl -a | grep temperature

in the shell. Note those values and boot to your BIOS and compare the temps and fanspeeds the BIOS is reporting with the values TN is seeing. If they are the same, address the hardware with the high temps
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
My SuperMicro BIOS most certainly does have fan speed control. Its under Advanced tab, Hardware heath Configuration/System Fan Monitor

View attachment 35133

Your Supermicro BIOS does not have fan speed control. This is just exposing the information from the IPMI subsystem, which is what controls the fan speed.
 

ptyork

Dabbler
Joined
Jun 23, 2021
Messages
32
Your Supermicro BIOS does not have fan speed control. This is just exposing the information from the IPMI subsystem, which is what controls the fan speed.
Sorry to crap on a super-old super-micro thread here (trying to remind myself how to slow my fans back down after they went berserk), but Supermicro makes workstation and server boards. Workstation boards (e.g., X9DAi) do NOT have IPMI and thus have fan controls in the BIOS. Server boards (e.g., X9DRi) do have IPMI and no controls in the BIOS. I much preferred the BIOS based controls. IPMI fan control on these boards suck. Or blow. Or whatever. Benefit is remote access and control without reboot, but the implementation is so half-rate it's painful to work with.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
Sorry to crap on a super-old super-micro thread here (trying to remind myself how to slow my fans back down after they went berserk), but Supermicro makes workstation and server boards. Workstation boards (e.g., X9DAi) do NOT have IPMI and thus have fan controls in the BIOS. Server boards (e.g., X9DRi) do have IPMI and no controls in the BIOS. I much preferred the BIOS based controls. IPMI fan control on these boards suck. Or blow. Or whatever. Benefit is remote access and control without reboot, but the implementation is so half-rate it's painful to work with.

Yes, but the X10DRI and X10SRM being discussed are server boards. You tell the difference because when you go to their product page, they have the terms "BMC" (the actual thing that handles the fans) and "IPMI" (the user interface). Supermicro also makes gaming boards and desktop boards, in addition to workstation boards, but we're not talking about any of those either.

The main problem with fan controls on the X9 IPMI is that Supermicro's boards are primarily designed to work in Supermicro rack mount chassis, and are pre-optimized towards that use case. This is hardly limited to Supermicro, though. Dell and HP both have weird fan speed issues when stuff like adding certain card configurations are added. It's very tweakable, but not as flexible as it could be.

I would note that when I first came to these forums, they were very heavy on users with crappy AMD APU's and janky desktop mainboard setups. My shop has been building bespoke custom servers based around Supermicro mainboards for probably around two decades, and I brought along with me a push towards server-grade boards, and when I finally wrote the original Hardware Recommendations thread, I pushed Supermicro -- hard. There are so many benefits to server-grade boards that some mild difficulties with fan control for people who want quiet servers is really not a big deal. It involves arcane knowledge, but so does SAS enclosure status, or crossflashing HBA's, or 10 Gig ethernet networking. That's what the forums are here to help out with.
 

C8Z06

Cadet
Joined
Sep 18, 2022
Messages
4
Thanks for this thread. I had the same issue, logged in with IPMI, and was able to change the setting from max to optimal and it's doing great now! Thanks!
 

MarcAurel

Cadet
Joined
Aug 7, 2023
Messages
5
er. Supermicro isn't all that's available. Either way, you should find fan control in your BIOS. My tower was running warm under auto mode, and switched it to Maximum - giving me another 10C. Look up the documentation for your board and figure out the speed control. Airflow? Make sure your pushing lots of air cross those platter boxes...
I'm sitting at 30C in ambient temp... I keep my drives at 128 (Minimum power usage without Standby (no spindown).

I'd say if your running some hot temps, take a look at your cooling situation...
I am looking for server grade Mainboard manufacturer other than Supermicro, ITX to ATX size. Could you elaborate what else there is available? The cheaper, the better.
 

ptyork

Dabbler
Joined
Jun 23, 2021
Messages
32
I am looking for server grade Mainboard manufacturer other than Supermicro, ITX to ATX size. Could you elaborate what else there is available? The cheaper, the better.
ASRock Rack comes to mind. Never used them, but they check all the boxes. Server-grade components, chipsets, standard ITX through ATX, IPMI, etc.


ASUS also makes a line of server motherboards with similar capabilities. Again, no personal experience.

 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
As a current user of a asrock rack board (Mini XL!) I’ll add that having an intermediary like iXsystems is really important if you want any kind of good customer support. The stories re Asrock ghosting folk post-sale are legend, whereas iXsystems offers AMAZING customer support.

If you want to buy a motherboard w/o an intermediary like iXsystems, I suggest saving bucks, grey hair, and acquiring something from SuperMicro - new or used. Older gear like my favorite dedicated SOHO storage server motherboard (x10sdv-2c-7tp4f) is still being sold for reasonable money.
 

MarcAurel

Cadet
Joined
Aug 7, 2023
Messages
5
As a current user of a asrock rack board (Mini XL!) I’ll add that having an intermediary like iXsystems is really important if you want any kind of good customer support. The stories re Asrock ghosting folk post-sale are legend, whereas iXsystems offers AMAZING customer support.

If you want to buy a motherboard w/o an intermediary like iXsystems, I suggest saving bucks, grey hair, and acquiring something from SuperMicro - new or used. Older gear like my favorite dedicated SOHO storage server motherboard (x10sdv-2c-7tp4f) is still being sold for reasonable money.
Have you needed the customer support a lot with your NAS? How expensive is it for 1 device in a freelancer office?
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
Have you needed the customer support a lot with your NAS? How expensive is it for 1 device in a freelancer office?
Hah.....

So, the first series of Motherboards associated with the mini and Mini XL was made by Asrock Rack (C2750D4i), which used a Avoton 2000-series embedded CPU from Intel. A number of issues plagued the board, including a bad Asrock BMC setting causing the local config flash to get nuked (by writing to it continuously). Most of these "teething" issues were resolved by the time I bought my Mini XL. iXsystems had also allegedly spent a lot of time and software resources to make the board reliable.

However, there was a industry-wide issue with the 2000-series CPUs courtesy of Intel, in the form of AVR 54. This bug was essentially a CPU flaw, which, with time, led to NAS' becoming un-bootable. Intel pledged to make things right via their customers, and allegedly everyone affected by a AVR54 bug could get their motherboard replaced within a certain time period.

However, the support that Asrock Rack offered was spotty at best, with people exchanging notes here, at STH, and elsewhere on how to contact whom at Asrock Rack to actually move their motherboard replacement process along. Thankfully I didn't have to, as I contacted iXsystems when my motherboard died and their support was simply awesome. I had to replace the motherboard multiple times, as some replacements were older returns or not properly repaired. I finally got a good model and could not be happier (revision 1.03 IIRC).

IxSystems most definitely lost money on me as a customer thanks to the multiple repairs, support exchanges, etc. Compared to folk who relied on Asrock Rack, I made out like a bandit, and all of that for free. If iXsystems offered the motherboard I wanted for sale from SM, I would have happily paid a significant premium to acquire that vs. buying it from WiredZone. The few times I contacted SM, (for issues that turned out to be benign) the support was excellent, though not as fast as iXsystems.

Based on what I have read here, at STH, and Reddit, I would NEVER buy a Asrock board from anyone but iXsystems unless I had zero expectation re: customer support (i.e. used, out of warranty, and not mission-critical). While iXsystems has the recurring transaction relationship with Asrock Rack to motivate them to play nice, a single-system customer like me would be undoubtedly be left in the rain if anything went wrong. No thanks.

That said, the C2750D4i rev. 1.03 is still fully-functional and makes a for a great compact Mini-ITX board that can handle a lot of drives. So I understand why iXsystems chose this board and sank so many resources into it. The current Mini XL apparently uses the A2SDi-H-TF from SuperMicro though I cannot confirm that from personal experience.
 
Last edited:

MarcAurel

Cadet
Joined
Aug 7, 2023
Messages
5
Sounds like it turned out ok for you at the end. I will stick with SM boards after your experience you described with AR. The SM A2SDi-H-TF looks like a really really interesting pick! More on the expensive side though and currently not available in Germany.
- I wonder if the Intel Atom C3758, 8C/8T, 2.20GHz (with fairly low clock speed) is a good pick for a TrueNAS, since SMB benefits a lot from single-thread performance.

The guides on this forum often recommend to avoid the ASUS server motherboards at all cost.
- I wonder why though, as with consumer motherboards ASUS is still among the top 3 manufacturers. Have you had any experience with ASUS too? @Constantin
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,828
No experience with asus, sorry.

As for great storage-oriented boards, also consider the X10SDV-2C-7TP4F. For $500 you get a flex atx board with two pcie 3.0x8, two SFP+, 20 (?) SATA slots, two SATADOMs, etc. Fast 2 core CPU makes for excellent SMB performance. For storage only, this is pretty unbeatable.
 
Top