Can't stagger spin-up of WD Red drives with LSI SAS 9211-8i

[hexdriver]

While @wintermute000 is trying to disable staggered spin-up, I'm trying to get it enabled. Here are the relevant pieces:

• FreeNAS 9.3-RELEASE
• SuperMicro X10SRi-F motherboard
• LSI SAS 9211-8i controller flashed to P16 IT-mode firmware
• 4x WD30EFRX HDs (to be expanded to 24 soon)
• NORCO RPC-4224 chassis

The WD Red drives normally spin up when power is applied. I can't prevent that by leaving pin 11 of the power interface floating (power comes through the NORCO backplanes). Also, WD's alleged method for enabling Power-Up in Standby (PUIS) using a jumper apparently doesn't work with Red drives.

I have, however, been able to enable PUIS on the drives using the HDAT2 utility. But once I do that, the LSI configuration utility can't recognize the drives and FreeNAS won't finish booting. FreeNAS booting hangs with an endless sequence of this message:

Code:

(probe0:mps0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 length 36 SMID 110 terminated ioc 804b scsi 0 state c xfer 0

Other points to consider:

• I currently have the spin-up delay set to 1 second in the LSI configuration. Max targets to spin up = 2. I've tried setting the LSI "Boot support" option to all of its possibilities without success and currently have it set to "Enabled OS only".
• Attaching the drives (with PUIS enabled) directly to the motherboard causes them to spin up at boot. In other words, the motherboard's controller can recognize and spin them up just fine.
• At one point I had PUIS disabled on the drives and booted into a portable Windows 8.1 environment. I spun down the drives manually using an HD utility. Then, when I tried to access the drives again, they spun up in a staggered fashion. If I enable PUIS, however, and then boot into Windows, it can't see the drives at all.

What am I missing? Is it not possible to get staggered spin-up with this combination of hardware?

 

[cyberjock]

Booting up with staggered spinup enabled requires that the hard drive, the backplane AND the controller it is attached to support it. I'm somewhat convinced that your configuration won't support this setup since you are getting the INQUIRY errors. The hard drive should have been spun up during LSI controller initialization (it should send the commands during the POST LSI init screen).

Spinning down disks manually and accessing them (which spins them up) doesn't provide a data point that is useful at all for staggered spinup.

What *should* happen, normally, is that the hard drive powers on and the disks firmware has the setting set that basically says "init everything, but don't spin up the disk". Then during POST initialization the controller polls the PHYs (SAS/SATA connectors) at the interval you specified in the controller, and each disk spins up appropriately. Then the controller sometimes polls all the disks a second time to ensure all is good and the bootup sequence continues. To the OS, the staggered spinup is totally invisible as the disks will (or at least should) be spun up and functional regardless of if you use staggered spinup or not.

PUIS is similar (but not the same, and should not be confused with staggered spinup). With PUIS the disks have the firmware set to power-on and go to standy mode. No tests are done, no spinup, nothing. Basically they are totally controlled by the controller they are attached to and are expected to do very little on their own without confirmation from the controller they are attached to. The OS generally doesn't issue these commands, but the driver your OS uses may tell the controller to issue the "wake up" command.

Controllers that don't support staggered spin (like most on-motherbard controllers) will often poll all of the PHYs immediately as part of the controllers POST. This effectively disables staggered spinup.

But controllers that support PUIS are few and far between from my experience, and I highly doubt the LSI HBAs people use for FreeNAS will support it. It's really unlikely that any standard HBA will support it. Most onboard controllers I've seen, either accidentally or by design (not sure which), wake up all hard drives with PUIS enabled at the same time and the behavior cannot be changed.

So what I get from what you showed above is:

Staggered Spinup is not functional with your hardware. Not sure what the limitation is, but my guess is your backplane. I have that exact chassis, and it has no extra frills, features, etc. I would argue it doesn't even have minimum features as the flowrate of air through the chassis is extremely poor and my hard drives overheat pretty easily. (I have a long thread where I had to replace their fans with Supermicro fans to prevent overheating the hard drives, and I am not alone with that behavior.)

PUIS aren't supported on your LSI, and Windows is clearly not able to wake them up.

So I don't think you have any options for staggered spinups of any kind without ditching the Norco Chassis. I bought a used Supermicro chassis (dual redundant PSUs and everything) from ebay for $350. If you shop around you can find them for cheap. I'd highly recommend you invest in one if your intention is to use this server long-term. I know people shy away from ebay for server stuff, but the price is so low, and the quality of Supermicro is so high, that I consider it a very good investment.

 

[DearestDreamer]

I'm building my own first server and I'd like to utilize this feature, I have a similar hardware setup:
X10SRL-F
836BA-R920B 3U
The backplane is a BPN-SAS-836A which reads: 3U 6Gbps Backplane Supports 16x3.5\" SAS/SATA HDD/SSD w/ iPass Connectors
It's basically 4 SFF8087 connectors that each connect to 4 SAS/SATA drives.

I was thinking of going with 2x IBM m1015 or 9211-8i, but looking at this I'd prefer investing in a HBA that can handle staggered spin-up. Am I correct to assume then that this is the only component standing in my way to do so? Would you perhaps know of a controller that does and is supported by FreeNAS, cyberjock?

Did you ever figure something out, hexdriver?