NAS - Reducing Power Consumption

[Harman]

Hey folks,
I recently built my first NAS from some spare computer parts lying around.

Here are the specs:
Mobo - Asus P8P67
CPU - Intel® Core™ i7-2600K
RAM- 16GB - RipjawsX 4GB DDR3 2133MHz x4
HDD - RaidZ1 - Seagate Ironwolf 10tb 7200rpm 6gb/s ST10000VN0004 x4
SSD - Temp storage - Samsung SSD 840 PRO 512gb
PSU - rm850x 80+ Gold

On idle my system consumes 63w. I've seen folks with raids that consume around 20w on idle.

Question: How can I improve the power efficiency of my NAS?

1. I'm assuming the most power consumption is from the HDDs. I've set the "Advanced Power Management" option on the seagate disks to "Level 128 - Minimum power usage without Standby (no spindown)". I'm guessing I could probably do better with setting it to "Level 1 - Minimum power usage with Standby (spindown)". But I've heard it isn't good to let the disks spin down constantly. What are your thoughts on this?

2. Again the system was just built from parts I had laying around, and most were intended for gaming. Are their any "significant" (at least 20w) savings in power I can achieve by replacing one of my components?

Thanks in advance for your time!
- Harman

 

[John Doe]

My experience to lower the consumption:
lower amount of hardware e.g. instead of using 4x4gbram go with 1x 16gb
avoid further hardware like further HBAs, NICs etc and use onboard connections
enabling higher C states of the CPU

regarding 1)
I decided to use also level 128 for my spinning rust, as I think that it would be more costly to replace an HDD than the advantage in power saving.

regarding 2)
I do not think that new hardware would be more economic than your existing in view of 2-3 years of usage, considering your actual hardware is for free.
I propose to check the C states of your CPU and to verify freenas is using them.
make sure tunetables is working. This lowered my consumption ~10w
check in the BIOS/UEFI if you can turn of unused mainboard controllers e.g. sound, serial etc.

 

[Harman]

Thanks John for the info!

I can definitely reduce my mem from 4 sticks to 2 (will just need to keep an eye out for deals).
I will also take a look at turning off some of the unneeded mobo controllers.
But first I'm going to try updating the CPU c-state as it seems like the biggest win.
So I hope I did this correctly...

Added the following entries to Tunables:

Variable Value Type Enabled
dev.cpu.0.cx_lowest C2 sysctl yes
dev.cpu.1.cx_lowest C2 sysctl yes
dev.cpu.2.cx_lowest C2 sysctl yes
dev.cpu.3.cx_lowest C2 sysctl yes
dev.cpu.4.cx_lowest C2 sysctl yes
dev.cpu.5.cx_lowest C2 sysctl yes
dev.cpu.6.cx_lowest C2 sysctl yes
dev.cpu.7.cx_lowest C2 sysctl yes

Determined that C2 was the lowest c-state by running:

root@freenas[~]# sysctl dev.cpu | grep cx
dev.cpu.7.cx_method: C1/hlt C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.7.cx_usage_counters: 16513 241444 0
dev.cpu.7.cx_usage: 6.40% 93.59% 0.00% last 10408us
dev.cpu.7.cx_lowest: C2
dev.cpu.7.cx_supported: C1/1/1 C2/2/80 C3/3/104
dev.cpu.6.cx_method: C1/hlt C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.6.cx_usage_counters: 17109 245470 0
dev.cpu.6.cx_usage: 6.51% 93.48% 0.00% last 8138us
dev.cpu.6.cx_lowest: C2
dev.cpu.6.cx_supported: C1/1/1 C2/2/80 C3/3/104
dev.cpu.5.cx_method: C1/hlt C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.5.cx_usage_counters: 17920 242074 0
dev.cpu.5.cx_usage: 6.89% 93.10% 0.00% last 14095us
dev.cpu.5.cx_lowest: C2
dev.cpu.5.cx_supported: C1/1/1 C2/2/80 C3/3/104
dev.cpu.4.cx_method: C1/hlt C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.4.cx_usage_counters: 16937 237479 0
dev.cpu.4.cx_usage: 6.65% 93.34% 0.00% last 18448us
dev.cpu.4.cx_lowest: C2
dev.cpu.4.cx_supported: C1/1/1 C2/2/80 C3/3/104
dev.cpu.3.cx_method: C1/hlt C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.3.cx_usage_counters: 21430 264294 0
dev.cpu.3.cx_usage: 7.50% 92.49% 0.00% last 1592us
dev.cpu.3.cx_lowest: C2
dev.cpu.3.cx_supported: C1/1/1 C2/2/80 C3/3/104
dev.cpu.2.cx_method: C1/hlt C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.2.cx_usage_counters: 29489 262846 0
dev.cpu.2.cx_usage: 10.08% 89.91% 0.00% last 2152us
dev.cpu.2.cx_lowest: C2
dev.cpu.2.cx_supported: C1/1/1 C2/2/80 C3/3/104
dev.cpu.1.cx_method: C1/hlt C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.1.cx_usage_counters: 17719 243776 0
dev.cpu.1.cx_usage: 6.77% 93.22% 0.00% last 1910us
dev.cpu.1.cx_lowest: C2
dev.cpu.1.cx_supported: C1/1/1 C2/2/80 C3/3/104
dev.cpu.0.cx_method: C1/hlt C2/mwait/hwc/bma C3/mwait/hwc/bma
dev.cpu.0.cx_usage_counters: 27274 316316 0
dev.cpu.0.cx_usage: 7.93% 92.06% 0.00% last 206us
dev.cpu.0.cx_lowest: C2
dev.cpu.0.cx_supported: C1/1/1 C2/2/80 C3/3/104

Please let me know if everything looks good! (Do i need to reboot?)

Thanks,
Harman

 

[sretalla]

You're unlikely to get much better without replacing the CPU with a lower TDP model and the spinning disks to 5400 RPM drives.

FreeNAS doesn't really spin down disks anymore (changed several versions back), so whatever settings change you do, it's still going to be spinning them at that 7200 RPM speed, which wi where most of your power goes.

Do let us know if the previous suggestions produce a reduction (and how much) though.

 

[John Doe]

I agree, you wont get much savings. Your CPU is almost 8 years old.

just calculate: how much will new hardware cost and how much will you spend to keep the new system running VS how much do you spend right now for energy consumption. calculate it over 5 years and 10 years.
just an example:
existing system non recurrent cost = 0 USD
existing system power consumption 1 year (60w; kwh=0.2 USD)=105.12USD
New system non recurrent cost=500USD
New system recurring cost 1 year (30w;kwh=0.2USD)=52.56

Cost for existing system over 5 years= 526.6USD
Cost for new system over 5 years= 762.8USD

Cost for existing system over 10 years=1051.2USD
Cost for new system over 10 years=1025.5USD

 

[Ericloewe]

CPU with a lower TDP model

That doesn't lower idle power. Newer CPUs do improve idle power, but it really has nothing to do with TDP.

 

[Chris Moore]

Are their any "significant" (at least 20w) savings in power I can achieve by replacing one of my components?

Not by replacing just ONE component. Like John Doe said:

I agree, you won't get much savings. Your CPU is almost 8 years old.

One thing that is going to save you electricity cost is getting into a new CPU, and RAM and that will require a new system board. This is something you should be looking into changing due to age, even if power was not an issue. At eight years, this system is definitely ready and you certainly should not try to use it beyond ten years. Replacing those components could bring your power consumption down significantly. I went from a socket 1366 Xeon system that was using 160 watts, not counting the drives, to a socket 1155 Xeon system that was only using around 70 watts, not counting the drives. Your savings could be even more if you move to a low power processor, but that depends on how you use your system. What processing requirements do you have? Is the system transcoding Plex video for example?
Also, like @sretalla said, you should look at, getting slower RPM drives. There is very little reason to have 7200 RPM drives and they do burn more power and generate more heat, but I am guessing that those are relatively new drives, so that is just something to think about for the future.

 

[Harman]

After a few days of applying the cpu c-states, the server was actually consuming 5-6 more watts!
I've reverted those changes for now.

There is very little reason to have 7200 RPM drives and they do burn more power and generate more heat, but I am guessing that those are relatively new drives, so that is just something to think about for the future.

FreeNAS doesn't really spin down disks any more (changed several versions back), so whatever settings change you do, it's still going to be spinning them at that 7200 RPM speed, which wi where most of your power goes.

I hope that's not the case, I spent way too much money on those drives! They are advertised as "superior" NAS drives

Your CPU is almost 8 years old.
just calculate: how much will new hardware cost and how much will you spend to keep the new system running VS how much do you spend right now for energy consumption. calculate it over 5 years and 10 years.

That's true, I won't see much of a savings for 5 years, assuming my next build will use a server specific cpu (xeon) and mobo with fewer but higher capacity memory modules. (Very nice pricing analysis by the way!)

but that depends on how you use your system. What processing requirements do you have? Is the system transcoding Plex video for example?

Currently I'm only using the the server for
* file sharing
* plex (hd movies/music/photos)

I'm thinking of setting up a home surveillance system as well, this project will probably give me a good estimate on my future hardware specs and costs.

Thank you so much John, sretalla, Ericloewe & Chris for you're suggestions and recommendations! It looks like the smart thing to do for now would be wait until my use's for the NAS are more clear before purchasing server friendly upgrade parts.

 

[Constantin]

FWIW, the ancient C2750D4I board that ixSystems uses in the Mini and MiniXl is excellent re: power consumption and can handle up to 12 SATA drives w/o issues. Great system for a home NAS. Supermicro shipped similar boards, some even with a HBA, IIRC.

 

[joeschmuck]

I went into a phase where I would do everything possible to save power at any cost, so long as it didn't cost me money. C-States were a factor but one of the major contributors besides sleeping the hard drives was under-clocking the CPU, motherboard bus speeds, RAM, etc... I was not transcoding video content at the time and it was just an experiment. If you know what you are doing then you can save some power but there is a performance penalty and you also need to test your system for stability. The 7200 RPM drives eat up more power than a 5400/5200 RPM drive and the performance in your application is going to be the same, but that purchase has already been done. Also the faster the hard drive, the noisier it tends to be. I would recommend you not worry about energy consumption as much as sound levels, cooling, and longevity. Here is why I say this... Lets say you have a system running at 70 Watts of power on average for 1 month (50400 watts total). Your power costs you 11 cents per kilowatt. Total cost is 54400 * .11 = $5.98/month. It's not free now is it. Lets say you could shave you consumption down 20 watts on average (that is very optimistic by the way) so you are now at 36000 total watts times 11 cents per Kw = $3.96 which is a grand savings of $2.02. Lets say your system consumes 100 watts/hr (72000 watts total for a month) then that will cost you 11 cents * 72.000 = $7.92 month.

For each hard drive that you can sleep you can gain 3 to 5 watts of power, however when you spin them up they consume a bit more in athe form of a power surge, additionally constant powering up the drive motor will shorten the motor life and tends to make your drives fail early. Running them 100% of the time actually extends their life. This is where you start calculating the difference of a drive lasting 3 years as opposed to 6 years (just an example).

My point here is unless your system is rarely used, you should instead focus on a well cooled system to ensure longevity and if silence is something you like, then that too. Small and silent generally do not go together unless you have all SSD and plan out your cooling very very well.

As you said, before purchasing anything new you should figure out what your system will be used for.

Good luck in whatever venture you pursue.

 

[LimeCrusher]

Currently I'm only using the the server for
* file sharing
* plex (hd movies/music/photos)

I'm thinking of setting up a home surveillance system as well, this project will probably give me a good estimate on my future hardware specs and costs.

Thank you so much John, sretalla, Ericloewe & Chris for you're suggestions and recommendations! It looks like the smart thing to do for now would be wait until my use's for the NAS are more clear before purchasing server friendly upgrade parts.

Yes, figure out precisely what you want to do with your NAS and select components accordingly to match your requirements. This is NAS building 101. If you envision a growth of your usage, check that you can upgrade components (put more RAM, connect more disks, upgrade the CPU through compatible sockets and so on ...).

FWIW, the ancient C2750D4I board that ixSystems uses in the Mini and MiniXl is excellent re: power consumption and can handle up to 12 SATA drives w/o issues. Great system for a home NAS. Supermicro shipped similar boards, some even with a HBA, IIRC.

While those motherboards are great (embedded super low power Intel Atom C, lot's of SATA connectivity), my feeling is they are only adapted to pure file sharing and low workloads. If you're planning on using Plex and transcoding 1080p video files, the CPU may struggle. The embedded aspect has a downside: you can't upgrade the CPU the day it becomes a limitation.
@Constantin loves (for good reasons) the very capable embedded Xeon D motherboards. Look for his posts and you'll find the references.