Staggered spin up

[hayer]

Hi.

So I started with this thread. So if figured I would get the HP SAS Expander that @danb35 suggested to support 24 drives.

Now my problem is the PSU. I'm pretty unsure how to "size" it. I've read the PSU sizing guide but would mean that I would need around 1200W without any headroom.

So now I'm sitting here wondering if I could use staggered spin up as I just need it to handle the spike at boot. Anyone got some experience or tips on that topic?

Current hardware is:
Intel i3-3220
SuperMicro X9SCM-F
4x 8GB Samsung ECC RAM
1x IBM M1015, flashed to IT
6x WD Red 1TB
6x WD Red 3TB
Corsaid CS750M (If I remember correctly, going to check in a couple of days)

And I'm want to add:
12x WD Red 3TB
HP SAS Expander


Looking for a solution that does not take up more than 1 PCI slot and is "low cost" compared to "buy 24port HBA" or "get a PSU, like this one; about $500". (And yes; I know I should get a proper PSU, so I'm looking at some of the Seasonic PSU that I see people recommend. But the 1000W+ are too pricey)



Edit, to mod:
Sorry for creating a new thread. Didn't cross my mind that I could just reply to/edit the other thread.

 

[depasseg]

What kind of case is this, that can handle 24 drives but not a set of proper PSU's?

 

[hayer]

Norco case. So all ATX PSUs are deemed "not proper" PSUs?

 

[Helios]

I suspect he's referring to something like this: http://www.legitreviews.com/images/reviews/589/antec_p190_007.jpg

 

[depasseg]

Norco case. So all ATX PSUs are deemed "not proper" PSUs?

If a single psu can't handle the expected load (without resorting to workarounds), then I would say no, that's not a proper setup.

But since it's designed to handle a single ATX PSU and 24 drives, does Norco recommend a certain power supply sizing? Maybe the one here is too conservative.

And yes, I was referring to using dual PSU's.

 

[maglin]

Most 4U enclosures with 25 HDD bays use 1200 watt PSUs but the key is 12V rail amperage.

For under $500 you can get an E16 4U enclosure with the HDDs you need along with the SAS2 backplane and 25 drive trays. Sounds better than spending so much money on a PSU and some crazy high port HBA.


Sent from my iPhone using Tapatalk

 

[hayer]

If a single psu can't handle the expected load (without resorting to workarounds), then I would say no, that's not a proper setup.

But since it's designed to handle a single ATX PSU and 24 drives, does Norco recommend a certain power supply sizing? Maybe the one here is too conservative.

And yes, I was referring to using dual PSU's.

I find it kinda hard to call staggered spin up a workaround. As I understand it one of the pros is that the drives(only some of them) spin up at the start of the boot - so if the server crashes and goes in a reboot-loop it won't kill your disks. But noted. Would you care to add some technical details to why not?

When it comes to "handling the expected load" I'm under the impression that the drives require quite a lot more of amp when starting up than when reading/writing. There is also just 2-3 users at most, but then again I can't guarantee that it sometimes will hit "worst case" and spin up all the disks. WD rate them at 4.1W for read/write, with 20% headroom that would be 4.1 * 1.2 * 24 = about 120W.

And sorry for sounding a bit offensive when replying to your first post, just interesting to learn new things so I get carried away.

Most 4U enclosures with 25 HDD bays use 1200 watt PSUs but the key is 12V rail amperage.

For under $500 you can get an E16 4U enclosure with the HDDs you need along with the SAS2 backplane and 25 drive trays. Sounds better than spending so much money on a PSU and some crazy high port HBA.


Sent from my iPhone using Tapatalk

The cheapest one I can get a hold of seems to be around $1000-2000 after adding shipping and taxes. And it might have been a bit unclear but I'm looking for solutions that does not include a 24port HBA that will cost me my first born child.

 

[Bidule0hm]

I'm under the impression that the drives require quite a lot more of amp when starting up than when reading/writing.

Not just an impression: https://forums.freenas.org/index.ph...drive-spin-up-peak-current.38885/#post-237516 :)

WD rate them at 4.1W for read/write, with 20% headroom that would be 4.1 * 1.2 * 24 = about 120W.

It's more along the lines of 30-35 W per drive during spin-up...

 

[danb35]

Not just an impression: https://forums.freenas.org/index.ph...drive-spin-up-peak-current.38885/#post-237516

But see https://forums.freenas.org/index.ph...spin-up-peak-current.38885/page-3#post-241620

 

[depasseg]

The fact that you are concerned about spin up power is a valid one. Whatever solution you use needs to handle every instance (not just boot, but spin up from idle for instance). Sometimes the HBA can help with this, but I can't see something baked into the OS FreeNAS helping, because the drives spin up way before the OS gets loaded. So whatever staggered spin up solution you use needs to handle both use cases of pre-boot and post-boot. Which means either something in hardware (like the HBA), or having adequate power so it's not an issue.

Oh, and you don't need a 24 port HBA. The SAS expander backplanes can take one SFF-8087 mini-SAS cable (which is really 4 ports/channels in one cable) connection and split out to handle all the drives.

 

[danb35]

This listing would be nearly turn-key; it would only require an HBA (which you already have) to replace the RAID controller (which you could probably sell for a couple hundred dollars). $665 shipped in the U.S. (not sure if you're in the U.S. since you don't indicate your location in your profile). Redundant 1400-watt power supplies, dual Xeons, 96 GB of RAM, IPMI. Doesn't include rails, though, so if you want to rack it you'd need to find a set.

Edit: Oh, and 36 bays rather than 24. Plenty of room for expansion.

 

[hayer]

It's more along the lines of 30-35 W per drive during spin-up..

Was an estimate for while doing work.

The fact that you are concerned about spin up power is a valid one. Whatever solution you use needs to handle every instance (not just boot, but spin up from idle for instance)

Well.. I have my doubts that I'll hit such a big peak if the server is already booted. But I guess that is a gamble bases on usage pattern.

not sure if you're in the U.S. since you don't indicate your location in your profile)

Would love to solve it that way.. but I'm in Norway. The used servers market is like next to nothing. Profile updated.


So anyone know if the m1015 flashed to IT combined with the SAS expander supports staggered spin up?


Edit:

I could buy this;
SuperMicro X8DTE-F
2x Intel Xeon X5550
48GB DDR EEC Reg
5x SuperMicro HBA AOC-SASLP-MV8
2x SuperMicro PWS-1K41P-QR
12x ST2000NM0011 Constellation ES 2TB

For about $2100 but then I would need to sell the old box minus the disks to cover the difference. And I have no clue how much(little) I can get for the old one.

Or this, as mentioned in my other post;
Chassis: Supermicro CSE 846TQ
Backplane: SAS846TQ rev. 3.1
Motherboard: H8DME-2 2.01A
Processor: 1x Quad-Core AMD Opteron 1.8GHz 2346 HE
RAM: 16GB 8x2GB DDR2 ECC Reg
Power Supplies: 2 x 900 W
HD Controller: 3x Supermicro SAT2-MV8
IPMI: SIM1U+ with AOC-USB2RJ45

For about $480 - but the controllers seems like they are where outdated. Obviously I don't want to use the included hardware, and the controllers seems like they can be considered "Not usable"..?


The main issue with number one is that it seems a bit to pricey, considering the HDDs are used and are probably dead within a year..? So that would mean I would have to get new drives as well.

 

[danb35]

but I'm in Norway. The used servers market is like next to nothing. Profile updated.

Bother. I've never really understood why computer gear is generally cheaper in the U.S. than anywhere else--it isn't like any of it is made here! I think the system I linked to is a better choice than your system #1 (and considerably better than #2), but I have no idea what shipping and such would cost. You really don't want a TQ backplane with 24 disks.

 

[hayer]

Bother. I've never really understood why computer gear is generally cheaper in the U.S. than anywhere else--it isn't like any of it is made here! I think the system I linked to is a better choice than your system #1 (and considerably better than #2), but I have no idea what shipping and such would cost. You really don't want a TQ backplane with 24 disks.

Yea, you mentioned in the previous thread regarding the "TQ issue" - the upside is that it would make me a pro when it comes to cable management, which I don't mind :p

When it comes to pricing I think that it boils down to just the number of people and easier/better mail system.

The price for your system shipped here, if that was an option, would probably be something like $1300, but that is a rough estimate - and all the sellers I've contacted on ebay doesn't want to ship over here for so small orders. Mainly because of the hassle of sending it and in case there is something wrong and I want to return it.


A friend also offered me a Corsair RM850 (which has 70.8A @ +12v) for a nice price- is that any usable?

 

[philhu]

My sc847 x8dtn+ board with 24 and 12 bay backplanes does delay spinup on its own for disks in the 12 bay backplane by 3-4 seconds

I see nothing in the bios of mb or lsi card that has that option but it sure does it.

 

[jgreco]

Once you pass a certain size, it is somewhat safer to stop linearly scaling the PSU for worst-case draw, but that's somewhere between the one and two dozen drive range. You can also set drives to power up in standby (PUIS). The real question is what happens over time if you happen to forget any magic tweaks you've made to manage current draw. If you set PUIS on all your drives, and figure out "yay I can get by running 12 drives on a 250 watt supply" (yes you can... maybe) and then you replace four drives over time with new drives and you forget to set PUIS, and one day you reboot the server, maybe the magic smoke comes out, possibly ruining 12 drives and your host system too.

In general, there is less danger when you're able to support a maximal load.

 

[cyberjock]

Just one thing I want to say about staggered spinup. The feature, for it to work properly, relies on the SATA/SAS controller supporting the function as well as the disks, and the combination being compatible. I've seen quite a bit of hardware that you'd expect to support it and work, but staggered spinup wouldn't work because of incompatibilities with the function between the controller and the disks. So I don't recommend going down this road as something as simple as a firmware update to your LSI card could suddenly disable the function.

To be honest, I'd say that anything over 800w should be fine for your setup.

 

[hayer]

Okey, turned out it was a Corsair AX850 -- just ordered it. To get the wheels turning. (Turned out it was a AX, not a RM) -- but they seemed pretty equal in the tech spec. Worst case scenario I put it in my workstation.

So should I just get an expander or?