Temperature monitoring

nick779

Contributor
Joined
Dec 17, 2014
Messages
189
Is there any way to get a simple readout of the system/HDD temperatures added to the GUI?

I have a script that runs nightly and emails me with the temps, but Id like to see an easier "at a glance" look through the GUI.

Thoughts?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
I'm fairly certain there's a feature request for this. Voice your support and hope it makes it into the rewrite that's coming with FreeNAS 10.
 
Joined
Jan 9, 2015
Messages
430
Use the IPMI port on your motherboard.
 

nick779

Contributor
Joined
Dec 17, 2014
Messages
189
Joined
Jan 9, 2015
Messages
430
10-4
 
Joined
Apr 15, 2013
Messages
8
An additional temperatures section in the GUI would be nice. Using the same graphing format, maybe one graph per physical cpu (with per core temps), system temps, and ones for hard drives. (Configurable to one graph all temps, or splitting out would be extra nice.)

...So does anyone have the correct feature request ticket where I can add the above? I couldn't find a specific one.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
I'd imagine if one existed it was probably shot down as "that's not a job for the OS". The problem is there's so many different types of hardware out there that there is a likelyhood of there being incompatibilities (or it not being supported at all for your hardware). The only really good option is to use the IPMI. It's designed to handle that because it's OS agnostic and tied to the hardware.

The right tool for the right job.
 

cdgonzalez

Dabbler
Joined
Dec 19, 2014
Messages
21
Is the argument that the IPMI interface is a better tool for the job? The existing SMART monitoring is done via the OS correct? Would a simply GUI monitor be that much further out of scope?

Edit:
Found https://bugs.freenas.org/issues/1565 - Feature Request - add a "SMART Status" page
 

Cellobita

Explorer
Joined
Jul 15, 2011
Messages
99
pfSense already does it, for the mobo and CPU cores (it also has access to SMART attributes, so disk temperatures are available) - perhaps it is possible to "borrow" the corresponding code and use it as a starting point.

pfSenseThermalSensors.png
 

Visseroth

Guru
Joined
Nov 4, 2011
Messages
546
I'm sorry to wake this thread up again, but system temps has been on my mind a LONG time. I've done digging and ran across this forum and thought I'd reply.

We have CPU graphs, memory graphs, heck even HDD activity graphs, those are about as necessary as CPU temp graphs. They are for informational purposes! HD, CPU and MB temp graphs, Heck even fan speed shows us how the hardware is doing in the machine, determining whether or not there is enough air flow or if there is some sort of problem, how the health of the hardware and supporting components are functioning, if a fan is going bad or if a heat sink is coming loose, ect. There are many reasons to monitor system wide temps and hardware, they give us insight into the health of the system and can give us a warning that system alerts may possibly miss. For instance if there is a time of day when the room that the system resides in is getting to warm or if under load if the system is having a hard time coping with getting rid of produced heat.
FreeNAS doesn't have the ability to just simply add an app so that someone can monitor the hardware being it uses a web interface and all. The lmsensors is already included in the CLI, why not a graph in the gui to show some history as well?
Yes, we can use things like splunk to monitor system loads, temps, ect. but not all of us are that savy. I've been trying to figure it out and I consider myself literate in a lot of ways, but apparently not enough to get splunk working decently and loading splunk into a VM adds more overhead than a simple graph ever would!

So, are temp graphs necessary? Absolutely!
Some may not think so, but then why the CPU, memory, HD, network, ect graphs? Because they provide valuable information. Another for instance would be, "Is the machine in question getting to full of dust and lint".
Not all of us put our machine is data center clean rooms. I'd love a data center style clean room but I also don't want the bill that comes along with it. I have better things to do with my money and if that means blowing out my machine every once in a while vs knowing it's clean all the time to save hundreds of $$ then I'm ALL over it!

Some may reply, "Well why don't you add it yourself?". Unfortunately I'm not a programmer nor do I have the knowledge necessary to do so, otherwise I would, then I'd write up a how to so others can enjoy the benifits of having system temp/fan speed graphs.
If I could afford it I'd just pay someone to do it. Heck, I wonder how many would donate to have someone write up such a feature.

Bottom line is, system health graphs are necessary. I'd love to have a health graph on everything including myself!

OK, enough of my ranting, but I hope to see this feature in version 10.

Thanks.
 

Visseroth

Guru
Joined
Nov 4, 2011
Messages
546
I'd imagine if one existed it was probably shot down as "that's not a job for the OS". The problem is there's so many different types of hardware out there that there is a likelyhood of there being incompatibilities (or it not being supported at all for your hardware). The only really good option is to use the IPMI. It's designed to handle that because it's OS agnostic and tied to the hardware.

The right tool for the right job.

Easy fix would be no detected sensors = no graphs
 
Joined
Apr 15, 2013
Messages
8
Sometime after 9.3 final came out last year, I dug into the code and was able to get a semi working graph system working. Problem was, it was all in dev as I could never figure out how to build my code into a working image to then mount to a real computer. I'll take another look this week to see if a) I still have my code and b) there was working build instructions available.

From memory, I essentially replicated the entire "system" tab under the reporting, and had it pulling from the correct sensors for temps. Given a few new tabs have popped up recently (targets and more zfs graphs) it shouldn't be too hard to replicate this much again.

(Apologies for poor grammar and spelling, using mobile...)
 

Seren

Dabbler
Joined
Feb 18, 2016
Messages
22
Sometime after 9.3 final came out last year, I dug into the code and was able to get a semi working graph system working. Problem was, it was all in dev as I could never figure out how to build my code into a working image to then mount to a real computer. I'll take another look this week to see if a) I still have my code and b) there was working build instructions available.

From memory, I essentially replicated the entire "system" tab under the reporting, and had it pulling from the correct sensors for temps. Given a few new tabs have popped up recently (targets and more zfs graphs) it shouldn't be too hard to replicate this much again.

(Apologies for poor grammar and spelling, using mobile...)

If you do locate your code, I'd love to see it. I poked around in the source at one point to see how hard it would be to add temperature graphs, but didn't get very far. Seeing what you did would be a nice headstart.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,458

Seren

Dabbler
Joined
Feb 18, 2016
Messages
22
Looks very nice, thanks for the work! I'm going to show my ignorance now and ask where the graphs appear--I'm sure they aren't in the web GUI anywhere (or are they?).

No, I didn't take to time to figure out how to integrate them into the main dashboard (and it would be wiped out every time the system was updated). That might a fun addition though.

Currently the graphs and rrd files are written into the same directory that the scripts are run from.
 
Last edited:

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,458
Ah, yes, that explains it. Thanks for clarifying. Easy enough to share out that directory and view the graphs from there. I'm running into problems with the scripts, though.

I created a dataset to put them in, did 'git clone https://github.com/seren/freenas-temperature-graphing' in that dataset, and set up the cron jobs as directed, adjusting the paths to suit. But every minute (and twice every five minutes), I get an email saying:
Code:
ERROR: /mnt/tank/scripts/freenas-temperature-graphing/temps-1min.rrd: expected 14 data source readings (got 9) from N


When I run the temps-simple.sh script, I get this:
Code:
[root@freenas2] /mnt/tank/scripts/freenas-temperature-graphing# ./temps-simple.sh
Sorry, user root is not allowed to execute '/usr/local/sbin/smartctl -a /dev/ada0' as root on freenas2.familybrown.org.
ada0 - 
Sorry, user root is not allowed to execute '/usr/local/sbin/smartctl -a /dev/ada1' as root on freenas2.familybrown.org.
ada1 - 
Sorry, user root is not allowed to execute '/usr/local/sbin/smartctl -a /dev/ada2' as root on freenas2.familybrown.org.
ada2 - 
Sorry, user root is not allowed to execute '/usr/local/sbin/smartctl -a /dev/ada3' as root on freenas2.familybrown.org.
ada3 - 
Sorry, user root is not allowed to execute '/usr/local/sbin/smartctl -a /dev/ada4' as root on freenas2.familybrown.org.
ada4 - 
Sorry, user root is not allowed to execute '/usr/local/sbin/smartctl -a /dev/ada5' as root on freenas2.familybrown.org.
ada5 - 
dev.cpu.0.temperature: 57.0C
dev.cpu.1.temperature: 57.0C
dev.cpu.2.temperature: 59.0C
dev.cpu.3.temperature: 59.0C
dev.cpu.4.temperature: 57.0C
dev.cpu.5.temperature: 56.0C
dev.cpu.6.temperature: 57.0C
dev.cpu.7.temperature: 57.0C
[root@freenas2] /mnt/tank/scripts/freenas-temperature-graphing# 


It may also be relevant that my system has a total of 14 drives: mirrored boot SATA DOMs (ada0-1), four more data disks on the onboard ports (ada2-5), and eight more data disks on an M1015 card (da0-7).

Edit: Removing the 'sudo' from the 'sudo smartctl...' line in temps-simple.sh did the trick for that script, but nothing for the others.
 
Last edited:

Seren

Dabbler
Joined
Feb 18, 2016
Messages
22
@danb35 Sorry for the delayed response. Since my system only has 4 drives, there's probably a bug with drives over 10 (since the error mentions 9 data-points). It might have something to do with text-parsing double-digit drive numbers, but that's speculation.

Could you login in as root, change your shell to bash, and run running the components manually? I'd start with the script that reads the SMART values:

Code:
/mnt/tank/scripts/temps-rrd-format.sh

If you get an error, could you run and post the results of this:
Code:
set -o xtrace
/mnt/tank/scripts/temps-rrd-format.sh
set +o xtrace


If the above works swimmingly, then could you do the same with the script that initializes and updates the rrd database:
Code:
/mnt/tank/scripts/temperature-monitoring/rrd.sh /mnt/tank/scripts/temperature-monitoring/temps-1min.rrd

and if there's an error, post the results of this:
Code:
set -o xtrace
/mnt/tank/scripts/temperature-monitoring/rrd.sh /mnt/tank/scripts/temperature-monitoring/temps-1min.rrd
set +o xtrace

Thanks. :)
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,458
Code:
[root@freenas2] /mnt/tank/scripts/freenas-temperature-graphing# ./temps-rrd-format.sh
44:45:49:49:40:40:42:42:25 0:25 0:33:32:31:32
[root@freenas2 /mnt/tank/scripts/freenas-temperature-graphing]# set -o xtrace
[root@freenas2 /mnt/tank/scripts/freenas-temperature-graphing]# ./rrd.sh temps-1min.rrd
+ ./rrd.sh temps-1min.rrd
ERROR: temps-1min.rrd: expected 14 data source readings (got 9) from N
[root@freenas2 /mnt/tank/scripts/freenas-temperature-graphing]# set +o xtrace
+ set +o xtrace

Edit: If I assume that the CPU temps are given first, and then the drive temps from ada0-ada6, the two drives that are reporting temps of "25 0" are my boot devices, two SATA DOMs. This seems to be confirmed by the output of temps-simple.sh:
Code:
[root@freenas2] /mnt/tank/scripts/freenas-temperature-graphing# ./temps-simple.sh
ada0 - 25C
0C 
ada1 - 25C
0C 
ada2 - 33C 
ada3 - 32C 
ada4 - 32C 
ada5 - 32C 
dev.cpu.0.temperature: 54.0C
dev.cpu.1.temperature: 54.0C
dev.cpu.2.temperature: 52.0C
dev.cpu.3.temperature: 52.0C
dev.cpu.4.temperature: 45.0C
dev.cpu.5.temperature: 45.0C
dev.cpu.6.temperature: 46.0C
dev.cpu.7.temperature: 46.0C
[root@freenas2] /mnt/tank/scripts/freenas-temperature-graphing# 
 
Last edited:
Top