GrumpyBear
Contributor
- Joined
- Jan 28, 2015
- Messages
- 141
While running extensive tests to validate my disks in preparation for a FreeNAS build using these guides:
Recommendations (TL;DR)
If you are using a SuperMicro X10-series motherboard in a pedestal (tower) case and desire adequate cooling for the hard drives with a minimum of noise:
Background
SuperMicro Motherboards are server grade motherboards and as such are designed for installation in rack-mounted cases in Data Centers. Adapting these boards for use in pedestal (tower) cases and with a goal of reducing system noise requires testing to ensure that the proposed thermal solution provides adequate airflow to keep the hard drives cool under high loads.
Electrical components tend to be more efficient at lower temperatures. Hard disks are electro-mechanical systems and the effects of high and low temperatures on their operation and predicted failure rate is not well understood. A widely quoted paper written in 2007 by a research team from Google "Failure Trends in a Large Disk Drive Population" which studied the failure trends in over 100,000 consumer-grade 5400-7200rpm drives deployed in their company suggesting that operating hard disks over 40 degrees Celsius may cause them to fail sooner than drives operating at lower temperatures.
Fans tend to be the largest source of noise in computer systems. In a data center where space is at a premium and high quality environmental controls are in place noise is acceptable and most rack-mounted systems have processors with passive heatsinks. In order to adequately cool these processors shrouds and baffles are utilized in chassis thermal designs with small, high rpm fans to move large volumes of air. Typically fan noise is directly proportional to rotational speed and the smaller a fan is the higher speed at which it must operate to move the same volume of air.
Computer fans typically operate at 12V DC. As they utilize DC motors the speed of the fans can be adjusted to a limited extend by lowering or raising the voltage (voltage control).
Most computer fans these days have a third connection (3-pin) that outputs a tachometer (TAC) signal (2 pulses-per-revolution). This TAC signal allows a controler on the motherboard to monitor the fan speed and detect a failed fan.
Higher end computer fans support varying the fan speed on the motor itself. These fans have a fourth lead (4-pin), in addition to the TAC signal, that accepts a Pulse-Width Modulated (PWM) signal from a controler on the motherboard to control the fan speed. The duty-cycle (percentage of on to off) of the PWM lead determines the speed at which the fan operates. At 0% duty cycle the fan is not running. At 100% duty-cycle the fan is running at its highest speed.
Like most rotating devices fans have inertia when stopped and require a certain amount of force to overcome this when started. To overcome inertia most fans are started by sending a PWM signal which starts at a higher duty-cycle and then is dropped down to the desired duty cycle. For this reason duty cycles lower than 30% are undefined.
Motherboard Capabilities
The SuperMicro X10SL7-F motherboard has 5 4-pin fan headers labelled FAN-1 through -4 and FAN-A. Each of the headers is rated at 1.5A (18W) with a combined rating not to exceed 3.0A
The BIOS and IPMI web interface has 4 possible fan modes; Standard, Full Speed, Optimal or PUE2 and, Heavy I/O. There is sparse information on the intended use of the fan headers and which fan mode should be used.
The motherboard monitors CPU, PCH, VRM and Memory temperatures using hardware built-in to the dies. There is additionally 2 thermistors monitored on the motherboard.
"RT1" is located between the SAS ports and the LSI2308 and is the "System" sensor
RT2 is located at between the ASPEED BMC and the I/O slots and is the "Peripheral" or "Aux" sensor.
Hardware Choices and Design Goals
I chose to use the Fractal Designs Define R4 case. This case was designed for the enthusiast gaming market segment and has a large volume and provision for up to six 120 or 140mm case fans. The case includes 2 140mm Silent Series R2 3-pin fans as well as a "fan controller" with a switch capable of operating 3 fans at 5, 7 or 12V (likely a resistive divider).
I replaced the 3-pin case fans with 4 Cougar Gaming 120mm Vortex PWM (4-pin) fans. The intent was to use large PWM controlled fans running at slow enough speeds to minimize noise and provide adequate cooling under load.
Extensive research on-line and on the Frequently Asked Question (FAQ) section of SuperMicro's support site led me to the initial supposition that the FAN-1 through FAN-4 headers had their PWM output controlled by CPU temperature and the FAN-A header had it's PWM output possibly controlled by another temperature sensor in fan modes other than Standard.
In order to drive the 4 case fans from the FAN-A header a 5-way 4-pin fan Splitter was used. The splitter was modified to extend the TAC leads from the additional fans back to the unused FAN-2 -4 headers:
as suggested by jgreco in the above thread I used this opportunity of having the disks at heavy load to validate my thermal and acoustical design.
Recommendations (TL;DR)
If you are using a SuperMicro X10-series motherboard in a pedestal (tower) case and desire adequate cooling for the hard drives with a minimum of noise:
- Use the "Optimal" fan mode.
- Use 120mm or 140mm PWM (4-pin) case fans connected to the FAN-A header.
- Connect the CPU cooler to one of the FAN-1 through FAN-4 headers.
- Use IPMItool to set the upper and lower thresholds for the fans.
- Verify that the disks are adequately cooled under heavy load.
SuperMicro X10SL7-F Motherboard
Xeon E31231v3 Processor
2 x 8GB Crucial ECC Memory
4 x WD 3TB Red HDD
4 x Seagate 3TB NAS HDD
Fractal Design Define R4 Case
Corsair HX650W Power Supply
Xeon E31231v3 Processor
2 x 8GB Crucial ECC Memory
4 x WD 3TB Red HDD
4 x Seagate 3TB NAS HDD
Fractal Design Define R4 Case
Corsair HX650W Power Supply
SuperMicro Motherboards are server grade motherboards and as such are designed for installation in rack-mounted cases in Data Centers. Adapting these boards for use in pedestal (tower) cases and with a goal of reducing system noise requires testing to ensure that the proposed thermal solution provides adequate airflow to keep the hard drives cool under high loads.
Electrical components tend to be more efficient at lower temperatures. Hard disks are electro-mechanical systems and the effects of high and low temperatures on their operation and predicted failure rate is not well understood. A widely quoted paper written in 2007 by a research team from Google "Failure Trends in a Large Disk Drive Population" which studied the failure trends in over 100,000 consumer-grade 5400-7200rpm drives deployed in their company suggesting that operating hard disks over 40 degrees Celsius may cause them to fail sooner than drives operating at lower temperatures.
Fans tend to be the largest source of noise in computer systems. In a data center where space is at a premium and high quality environmental controls are in place noise is acceptable and most rack-mounted systems have processors with passive heatsinks. In order to adequately cool these processors shrouds and baffles are utilized in chassis thermal designs with small, high rpm fans to move large volumes of air. Typically fan noise is directly proportional to rotational speed and the smaller a fan is the higher speed at which it must operate to move the same volume of air.
Computer fans typically operate at 12V DC. As they utilize DC motors the speed of the fans can be adjusted to a limited extend by lowering or raising the voltage (voltage control).
Most computer fans these days have a third connection (3-pin) that outputs a tachometer (TAC) signal (2 pulses-per-revolution). This TAC signal allows a controler on the motherboard to monitor the fan speed and detect a failed fan.
Higher end computer fans support varying the fan speed on the motor itself. These fans have a fourth lead (4-pin), in addition to the TAC signal, that accepts a Pulse-Width Modulated (PWM) signal from a controler on the motherboard to control the fan speed. The duty-cycle (percentage of on to off) of the PWM lead determines the speed at which the fan operates. At 0% duty cycle the fan is not running. At 100% duty-cycle the fan is running at its highest speed.
Like most rotating devices fans have inertia when stopped and require a certain amount of force to overcome this when started. To overcome inertia most fans are started by sending a PWM signal which starts at a higher duty-cycle and then is dropped down to the desired duty cycle. For this reason duty cycles lower than 30% are undefined.
Motherboard Capabilities
The SuperMicro X10SL7-F motherboard has 5 4-pin fan headers labelled FAN-1 through -4 and FAN-A. Each of the headers is rated at 1.5A (18W) with a combined rating not to exceed 3.0A
The BIOS and IPMI web interface has 4 possible fan modes; Standard, Full Speed, Optimal or PUE2 and, Heavy I/O. There is sparse information on the intended use of the fan headers and which fan mode should be used.
The motherboard monitors CPU, PCH, VRM and Memory temperatures using hardware built-in to the dies. There is additionally 2 thermistors monitored on the motherboard.
"RT1" is located between the SAS ports and the LSI2308 and is the "System" sensor
RT2 is located at between the ASPEED BMC and the I/O slots and is the "Peripheral" or "Aux" sensor.
Hardware Choices and Design Goals
I chose to use the Fractal Designs Define R4 case. This case was designed for the enthusiast gaming market segment and has a large volume and provision for up to six 120 or 140mm case fans. The case includes 2 140mm Silent Series R2 3-pin fans as well as a "fan controller" with a switch capable of operating 3 fans at 5, 7 or 12V (likely a resistive divider).
I replaced the 3-pin case fans with 4 Cougar Gaming 120mm Vortex PWM (4-pin) fans. The intent was to use large PWM controlled fans running at slow enough speeds to minimize noise and provide adequate cooling under load.
Extensive research on-line and on the Frequently Asked Question (FAQ) section of SuperMicro's support site led me to the initial supposition that the FAN-1 through FAN-4 headers had their PWM output controlled by CPU temperature and the FAN-A header had it's PWM output possibly controlled by another temperature sensor in fan modes other than Standard.
In order to drive the 4 case fans from the FAN-A header a 5-way 4-pin fan Splitter was used. The splitter was modified to extend the TAC leads from the additional fans back to the unused FAN-2 -4 headers: