I need some data points

[melbogia]

So I am looking to make a put together a NAS box. I'll be using E3 1230V3, 32 GB RAM, and 6 3TB WD REDs. My questions are:

Power: Can anybody chime in with 6 WD REDs and similar processor and tell me how much power their rig uses on idle? I am sure the fans take some decent amount as well, i.e, it's not negligible.

Raidz2 performance: This will be for general storage, backups (TimeMachine or other), Plex (Lets say I want to transcode 2 streams). I will have 1 raidz2 vdev. Can I realistically expect to get 2 HD streams and maybe a TimeMachine at the same time, without any hiccups to the streams?

 

[marbus90]

Should be around 50W idle.

I've heard of people running torrent seedboxes on such arrays which don't have trouble. 32GB RAM may be a bit over the top for a start. As for the CPU, either pick the E3-1220 v3 or the E3-1241 v3 - 1220 is the entry in the xeon world with very good performance (which I'm using in workstations) and 1241 is when you want a bit more performance. anything in between is not feasible imho.

My build would look like this:
Fractal Node 304
Seasonic G-360
ASRock MT-C224
E3-1241 v3
CT2KIT102472BD160B DIMM Kit

Mini-tower alternative:
Fractal Design Arc Mini or Define Mini
Seasonic G-360
X10SLL-F, X10SLL+-F, X10SLM-F. X10SLM+-F
E3-1241 v3
CT2KIT102472BD160B DIMM Kit

 

[jgreco]

The G-360 is underpowered for a Xeon with six drives. marbus90 will argue otherwise, and is technically correct in that it isn't actually overloaded, but a slightly larger supply is a better choice from a reliability perspective.

 

[marbus90]

s/underpowered/overpowered.

We've already established that the G-360 PSU will run at <20% load most of the time. Peak consumption during boot won't exceed 80%. That does fit my PSU sizing bill.

 

[jgreco]

It was already explained to you with numbers why that's wrong. You're welcome to size your PSU however you see fit, but "most of the time" is not the important metric in sizing a power supply.

Let's try a saner, based-in-reality-and-safety estimate.

We have to account for:

Base system (MT-C224 mainboard 15 watts, CPU 80 watts, memory 2 x 6 watts) -107 watts, peak consumption.
Fans - 15-30 watts.
Drives - 8 watts times 6 drives. This is idle current, NOT spinup current, and it is definitely additive ON TOP of the spinup. Which is ~144 watts, but could trend a bit higher.

That's 329 watts right there. However, you need to account for the fact that you shouldn't be peaking out a power supply, regardless of what the marketing glossies say. A 360 watt supply should never be asked to give more than 80% of its rated power, or around 290 watts.

Now, what marbus90 is going to say is that "oh but that's okay, see, even jgreco says 329 < 360". As an engineer, I solidly believe in derating of electronic assemblies, because we know that if you drive something all the way to spec, it is more likely to develop faults, and as an assembly is in use for years and it ages, its ability to sustain its rated spec may decline somewhat as well. So I really like to see the math that shows the system isn't likely to draw more than 80%. This also gives a lot of elbow room if someone plugs in a faulty USB device, or a fan seizes up, or something else bad happens, or when you need to temporarily attach some other drives for data migration purposes.

But, the truth is that the 360 is *probably* going to be okay, even though peak potential current is over 80%.

However, there's really not much of a penalty to go with the 450 instead and be much more solidly in the comfort zone. You're attaching over a thousand dollars worth of electronics to the thing. Why take a chance?

That comes from my experience building these things professionally. I like the headroom and I don't like to blindly trust that Seasonic has actually built in sufficient capacity to cope.

So anyways, as you can see, marbus90 and I have a disagreement about the definition of "overpowered." You're free to analyze my explanation and either accept or discount it. I've shown my math and my reasoning.

 

[Ericloewe]

The real advantage of the G-450 are the modular cables, which make cable management somewhat easier.

 

[jgreco]

The real advantage of the G-450 are the modular cables, which make cable management somewhat easier.

An excellent point, a good modular cable design can be a real nice feature and helps to maximize airflow while making for less of a cabling nightmare.

 

[marbus90]

It was already explained to you with numbers why that's wrong. You're welcome to size your PSU however you see fit, but "most of the time" is not the important metric in sizing a power supply.

Read a few words more. There you see the word "peak" ...

Base system (MT-C224 mainboard 15 watts, CPU 80 watts, memory 2 x 6 watts ) -107 watts, peak consumption.
Fans - 15-30 watts.
Drives - 8 watts times 6 drives. This is idle current, NOT spinup current, and it is definitely additive ON TOP of the spinup. Which is ~144 watts, but could trend a bit higher.

Just nope. You're doing enterprise-grade calculations, not fincky home-user ITX builds.
Mainboard wouldn't be near 15W. In fact, the Mainboard and RAM and CPU together will not exceed 80W at the most. But I'll give you the 10A for safe measures. And that isn't exactly drawn in the moment the HDDs spin up. At that point the CPU isn't fully utilized.
Fans - 2x 92mm, 1x 120mm + 1x Intel boxed fan. 0.5A at most.
HDDs - 3TB Reds are at ~0.5A each. Plus 1.5A spinup current. Even if you assume that drives use both spinup peak usage rating and the standard usage _on top_ (which would mean they write bullshit when they note "peak usage 1.5A"), that's 12A.

Grand total on the 12V rail would be 22.5A.

The Seasonic G-360 has a rating of 30A. There. Not even 80% during theoretical peak load. And we didn't even consider that some of these amps isn't fed by 12V but by 5V/3.3V.

 

[jgreco]

Read a few words more. There you see the word "peak" ...

Which is also wrong, but my point was that "most of the time" is basically irrelevant.

Just nope. You're doing enterprise-grade calculations, not fincky home-user ITX builds.

<sarcastic>I forgot. The electrons work differently if they're at home.</sarcastic>

Mainboard wouldn't be near 15W. In fact, the Mainboard and RAM and CPU together will not exceed 80W at the most.

The only way to establish that as a fact is to actually put it on the bench and prove it with an actual power meter.

But I'll give you the 10A for safe measures.

10 amps of what? 12V? 5V? 3.3V?

And that isn't exactly drawn in the moment the HDDs spin up. At that point the CPU isn't fully utilized.

Really? So I'm a home user, I set my drives to spin down, then along comes a 3AM automatic task to refresh a Plex database, suddenly CPU surges to 100% as a bunch of Python launches off and all the drives on the pool spin up because of activity?

Oh, yeah, I'm sure you were assuming that the only time that you'd encounter a spinup event was during BIOS POST? But even there, sometimes the CPU is eating power for unexpected reasons.

Fans - 2x 92mm, 1x 120mm + 1x Intel boxed fan. 0.5A at most.

Really? In what fantasy world?


Ok, so I didn't have two 92mm fans and a 120mm readily available so I just used two 120's and an Intel fan. There it is. 1.07 amps - very near the lower number I mentioned (15W). And, horrifyingly, I caught the inrush current up at 2.72 amps, and a single fan stall at 2.39 amps. Unusual for me to underestimate but there we have it.

HDDs - 3TB Reds are at ~0.5A each. Plus 1.5A spinup current. Even if you assume that drives use both spinup peak usage rating and the standard usage _on top_ (which would mean they write bullshit when they note "peak usage 1.5A"), that's 12A.

Of course it's on top. The spinup current is always broken out since it is the dominant issue. I have no idea where you get 1.5A spin current from. For the WD Red's, it's 1.75A on the 12V according to their own tech docs. http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-800002.pdf I admit I usually just assume 2 or 2.1 so you do gain a little bit there.

Grand total on the 12V rail would be 22.5A.

The Seasonic G-360 has a rating of 30A. There. Not even 80% during theoretical peak load. And we didn't even consider that some of these amps isn't fed by 12V but by 5V/3.3V.

Too little to the important operational aspects, unfortunately.

But what you're failing to explain here is why it is that you feel it so necessary to shave down the size close to the capacity of the power supply. We went through this last time, complete with me calling B/S on your 1.5A spin current claim, which is still a dominant factor in all of this.

There's little harm in going up a little in size. There's tremendous harm to the system if your marginally-large-enough supply sags while spinning the drives. Please explain why you have such a thing about going for the smaller supply.

 

[marbus90]

<sarcastic>I forgot. The electrons work differently if they're at home.</sarcastic>

Point being not using a dozen 10krpm fans but very few, very low RPM fans.

10 amps of what? 12V? 5V? 3.3V?

12V as it is the main current used in systems. And I really overdid it with 10A here as my whole system (E3-1225 v3, 32GB RAM, 1 SSD) uses 70W at the wall socket with boinc and prime95 running at the same time, meaning ultimate CPU meltdown.

Really? So I'm a home user, I set my drives to spin down, then along comes a 3AM automatic task to refresh a Plex database, suddenly CPU surges to 100% as a bunch of Python launches off and all the drives on the pool spin up because of activity?

Oh, yeah, I'm sure you were assuming that the only time that you'd encounter a spinup event was during BIOS POST? But even there, sometimes the CPU is eating power for unexpected reasons.

Implies that there is an additional SSD to move the .system dataset to. for which there wouldn't be a free SATA port as we're discussing a system with 6 SATA ports and 6 HDDs. If that dataset is on the zpool, the drives don't spindown. Some bad drives are just parking their heads.

Really? In what fantasy world?
View attachment 7765
Ok, so I didn't have two 92mm fans and a 120mm readily available so I just used two 120's and an Intel fan. There it is. 1.07 amps - very near the lower number I mentioned (15W). And, horrifyingly, I caught the inrush current up at 2.72 amps, and a single fan stall at 2.39 amps. Unusual for me to underestimate but there we have it.

Those fans are included with the Node 304:
http://www.fractal-design.com/home/product/casefans/silent-series-r3/silent-series-r3-92mm
1x 140mm -> 0.2A
2x 92mm -> 0.07A each, 0.14A total
1x Intel Boxed -> 0.2A as per http://content.hwigroup.net/images/products_xl/166262/intel-socket-1155-boxed-cooler.jpg
Grand total for fans: 0.54A

Of course it's on top. The spinup current is always broken out since it is the dominant issue. I have no idea where you get 1.5A spin current from. For the WD Red's, it's 1.75A on the 12V according to their own tech docs. http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-800002.pdf I admit I usually just assume 2 or 2.1 so you do gain a little bit there.

We're debating 3TB Reds here. You'll see in that same exact datasheet that 3TB Reds are rated for 1.5A spinup current, I believe only the 5 and 6TB ones were rated 1.75A. And then there are the reports of users which claim that a HDD only uses 15W during spinup... well. Also the 3TB Reds were rated 5.4W -> ~0.5A, so I give in a bit with 2A per disk and be done.

But what you're failing to explain here is why it is that you feel it so necessary to shave down the size close to the capacity of the power supply. We went through this last time, complete with me calling B/S on your 1.5A spin current claim, which is still a dominant factor in all of this.

There's little harm in going up a little in size. There's tremendous harm to the system if your marginally-large-enough supply sags while spinning the drives. Please explain why you have such a thing about going for the smaller supply.

Because I was a Green IT nerd before it was cool, i.e. before VMware integrated DPM I searched for 3rd party solutions. It is known that the efficiency of PSUs drops drastically below 20% utilisation. As a NAS PSU will sit at or even below this load marking for 95% of the time, I do lay my focus on the 95th percentile for home systems. - whilst still satisfying your 80% ruling for peak loads. Theoretical peak power draws are already worked into the PSU design. I recall some PSUs running at 125% load for extended periods of time (not short bursts) and are capable of pushing 150% for bursts like inrush currents.

But well, there's also that difference in philosophy. In the EU (where I did my apprenticeship as IT system electrician and graduated with a rating of 91% (I didn't quite get that social part) just 5 years ago, worked with anything from 12VDC in cars to 400VAC 3ph plus radio/networking) we just know that stuff doesn't die from short bursts. We don't know your 80% load limit on breakers. We throw 16A at our 16A breakers and expect them to hold. Starting at 20A those breakers are allowed to actually break - after an hour. Cables are sized respectively.

 

[jgreco]

We're debating 3TB Reds here. You'll see in that same exact datasheet that 3TB Reds are rated for 1.5A spinup current, I believe only the 5 and 6TB ones were rated 1.75A.

You managed to find something other than the WD30EFRX, which is listed at 1.73A? Hey, honestly, I'm blind. Point me at it like I'm an idiot.

And then there are the reports of users which claim that a HDD only uses 15W during spinup... well. Also the 3TB Reds were rated 5.4W -> ~0.5A, so I give in a bit with 2A per disk and be done.

I'd really want to see that on a bench grade ammeter before applying a multiplicative factor (of * 6) to it. I'm sure some guy somewhere tried to monitor his system with a Kill-A-Watt and extrapolated that his drive was only using 15W spinning up.

Because I was a Green IT nerd before it was cool, i.e. before VMware integrated DPM I searched for 3rd party solutions. It is known that the efficiency of PSUs drops drastically below 20% utilisation. As a NAS PSU will sit at or even below this load marking for 95% of the time, I do lay my focus on the 95th percentile for home systems. - whilst still satisfying your 80% ruling for peak loads.

I can appreciate that, but I'm not going to tell someone something I can't test or prove. The conservative option is to figure out the worst case scenario.

PSU efficiency does drop below 20% utilization. So let's play the numbers game. And look, I've actually tried pretty hard here to source facts not estimates. So work with me, 'k?

and the data at http://www.techpowerup.com/reviews/Seasonic/G550/6.html

So that's output power: power measured as a sum of the amps times volts of all the DC supplies. A 60 watt system is operating at 16% of rated load on a 360W supply looks to be around 86% efficient, while that same system is only around 11% rated load on a 550W supply, which is 83% efficient. I've picked the 550 because I was able to find numbers for it - not because it's a good idea. I have to assume the G-450 would perform "better" but I am giving your position a head start.

The 360 watt supply supplying 60W at 86% efficiency is therefore consuming 70W at the wall. The 550 watt supply at 83% efficiency is consuming 72W at the wall.

That's TWO WATTS MORE due to the inefficiency of the larger supply.

At a cost of 14c/kWh, it would cost you about $12.26 over the next five years for that larger supply. Probably less if you went for the G-450. The "green" savings is exceedingly minimal.

Theoretical peak power draws are already worked into the PSU design. I recall some PSUs running at 125% load for extended periods of time (not short bursts) and are capable of pushing 150% for bursts like inrush currents.

In theory. But the typical PC power supply, even the high end ones, I'm sorry, I just don't have that sort of faith in them. I've seen them burn.

But well, there's also that difference in philosophy. In the EU (where I did my apprenticeship as IT system electrician and graduated with a rating of 91% (I didn't quite get that social part) just 5 years ago, worked with anything from 12VDC in cars to 400VAC 3ph plus radio/networking) we just know that stuff doesn't die from short bursts. We don't know your 80% load limit on breakers. We throw 16A at our 16A breakers and expect them to hold. Starting at 20A those breakers are allowed to actually break - after an hour. Cables are sized respectively.

You can do a similar thing here in the US. Derating of electrical is kind of a complex topic, you just have to think of the 20A as being the point at which a breaker is expected to eventually trip, while holding 16A indefinitely.

So the question comes down to, is it better to save the two watts, or is it better to build in a larger safety margin to protect your $1000++ NAS and the data stored therein?

I don't view that as a difficult decision, but obviously we reach different conclusions.

 

[Starpulkka]

(Warning) Do not use Seasonic G-360 if you going to have 6 HDD's. Here in Finland there has been raised an awarness of those molex-data adapters. There has been numerous close calls and one house burn, because of molex-sata adapter failure.

I even myself used molex-sata adapter on my WD Red 6TB and after a day i had weird hdd is lost messages in windows. Tryed and checked almost all possibilities, and then i remembered my own advice that WD hdds are picky for power. I changed adapter to real sata power and hdd broblems went away an instant.

Not sure is it about adapter quality or some other factor, but at this time i really do recommend to not use molex-sata adapter.

Show

http://www.amazon.com/review/R61VX6TRGF7GK/

Show

http://www.stevenhale.co.uk/main/2013/11/more-sata-adapter-fires/

 

[Ericloewe]

(Warning) Do not use Seasonic G-360 if you going to have 6 HDD's. Here in Finland there has been raised an awarness of those molex-data adapters. There has been numerous close calls and one house burn, because of molex-sata adapter failure.

I even myself used molex-sata adapter on my WD Red 6TB and after a day i had weird hdd is lost messages in windows. Tryed and checked almost all possibilities, and then i remembered my own advice that WD hdds are picky for power. I changed adapter to real sata power and hdd broblems went away an instant.

Not sure is it about adapter quality or some other factor, but at this time i really do recommend to not use molex-sata adapter.

Show

http://www.amazon.com/review/R61VX6TRGF7GK/

Show

http://www.stevenhale.co.uk/main/2013/11/more-sata-adapter-fires/

That's mighty weird, given that there's little to get wrong when making those things.

I can definitely recommend not chaining too many adapters, though.

 

[Bidule0hm]

The problem isn't the PSU, it's the adapters. When you buy in china something like 10 adapters for $1 then problems arises...

 

[Ericloewe]

The problem isn't the PSU, it's the adapters. When you buy in china something like 10 adapters for $1 then problems arises...

Still, I could probably make decent adapters and still profit at 10 cents a piece.

 

[Bidule0hm]

Yep, as always, if you want something right, do it yourself ;)

I think the best (and almost free) way to convert a molex into a sata is probably to solder a sata power cable of another dead/useless PSU in place of the molex but I understand that not everyone can (and want to) do soldering on his system :)

 

[Ericloewe]

For modular PSUs which do not come with enough SATA chains to fill all Molex/SATA outputs on the PSU, the manufacturer may also sell cables separately.

 

[Starpulkka]

First G-360 isnt modular. And it has only 4 sata power connectors. And yes 3.3volts has been talked enought on another forum. And yes theres also enough talk about electromigration. But why you dont spend few euros more that you get psu that has ready 6 sata connectors in first place, than you have to start buidling your own adapters and if something bad happens, you then can blame only yourself and not psu manufacturer.