Guide to setting up Grafana + InfluxDB Metrics

[Dravinian]

I have looked and there is no cpugraf - after I renamed it - took a moment as I had to set up SSH because show series that I attempted earlier was truncated.

My entire template is:

templates = [
"*.*.cpugraf.* .host.measurement.cpu.field",
"*.*.uptime.* .host.measurement.field",
"*.app env.service.resource.measurement",
"servers.* .host.resource.measurement.field*",
"*.*.load.* .host.measurement..term.field",
# # Default template
# "server.*",
]

I have other readouts in Grafana so I am guessing my other settings are ok.

 

[Patrick M. Hausen]

I don't quite get what you are attempting. All measurements generated by TrueNAS are prefixed with "servers."

 

[Dravinian]

I don't quite get what you are attempting. All measurements generated by TrueNAS are prefixed with "servers."

I am attempting to add the temps to a Grafana dashboard, I had hoped using the template would save me a lot of entries.

However, I can't even seem to get:

servers.freenas_local.cputemp.0.temperature

To provide me any data.

I am looking at the Grafana select methods and while I have tried several variations, this one is currently providing me no data:

"SELECT mean("value") FROM "servers.freenas_local.cputemp.0.temperature" WHERE time >= now() - 3h GROUP BY time(10s) fill(null)"

When I look at the Query Inspector:

request:Object
method:"GET"
url:"api/datasources/proxy/3/query"
params:Object
db:"graphite"
q:"SELECT mean("value") FROM "servers.freenas_local.cputemp.0.temperature" WHERE time >= now() - 3h GROUP BY time(10s) fill(null)"
epoch:"ms"
data:null
precision:"ms"
hideFromInspector:false
response:Object
results:Array[1]
0:Object
statement_id:0

When I compare this to a working query:

Object
request:Object
method:"GET"
url:"api/datasources/proxy/3/query"
params:Object
db:"graphite"
q:"SELECT last("temperature") FROM "ada0" WHERE ("resource" = 'disktemp') AND time >= now() - 3h GROUP BY time(2m) fill(null);"
epoch:"ms"
data:null
precision:"ms"
hideFromInspector:false
response:Object
results:Array[9]
0:Object
statement_id:0
series:Array[1]
0:Object
name:"ada0"
columns:Array[2]
0:"time"
1:"last"

I am assuming that my query is incorrect, but I am not sure why or what I should be querying even on the basic cputemp.0.temperature.

 

[Patrick M. Hausen]

I'm by no way an Influx expert, but this looks like your TrueNAS simply does not send the servers.freenas_local.cputemp.0.temperature series then. Maybe the collectd running on your TrueNAS cannot read the temperature data from the hardware you are using.

All I ever did was configure my Influx installation as the target and this template: "servers.* .hostname.resource.instance.measurement*"

Then I browsed the output of show series and toyed in Grafana with what I found there. If there is no data in your series, it's probably not sent.

Does your system have an IPMI interface? Observium via SNMP can be enriched with IPMI data and that shows a ton of stuff collectd/influxd/grafana doesn't.

 

[Dravinian]

Thing is, I was looking around online - found a thread on here:

In that thread the individual used this command:

SELECT * from "servers.freenas_local.cputemp-0.temperature" LIMIT 10

I tried that and it failed, but looking closer he has a cputemp-0, I have a cputemp.0 - so changing that....I get a result, both in the Grafana Jail and if I send this as a query via Grafana - the issue is that the data returned in Grafana is a presentable format.

Trying to graph CPU temp in Grafana

Hey guys, I'm trying to graph my CPU temp in Grafana. I've managed to get Freenas to dump all the data into Influx, but the data seems off or I'm using the wrong units. When I manually query influx, here's what I see: > SELECT * from "servers.freenas_local.cputemp-0.temperature" LIMIT 10...

www.truenas.com

 

[Dravinian]

Welp putting in this:

SELECT "value" /10-273.15 FROM "servers.freenas_local.cputemp.1.temperature" WHERE time >= 1666471023000ms and time <= 1666474153000ms LIMIT 10

Provides an output, which is enough to build a dashboard off of I think.

 

[Patrick M. Hausen]

The ".0" vs. "-0" is a feature of the "graphite separate instances" setting. If you enable this and expand your template to have one more hierarchy level like mine above, you can query for instances or do e.g. an average across all of them etc.

To elaborate - the question is if you want a resource "cputemp-0" and another resource "cputemp-1" and so forth. Or a resource "cputemp" with instances 0, 1, 2, ...

 

[Dravinian]

The ".0" vs. "-0" is a feature of the "graphite separate instances" setting. If you enable this and expand your template to have one more hierarchy level like mine above, you can query for instances or do e.g. an average across all of them etc.

To elaborate - the question is if you want a resource "cputemp-0" and another resource "cputemp-1" and so forth. Or a resource "cputemp" with instances 0, 1, 2, ...

Sorry, it took me a minute to catch up to where you were.

Yes, adding .instance. to the server collection template did the trick for me and allows me to more easily gather the information.

So cheers for that!

 

[toxikat]

Thanks for the guide! I've got things mostly up and running at this point, but I'm struggling a bit with the data collection (and subsequently the grafana part).

I've got my template like this:

Code:

templates = [
    "*.*.cputemp.*.host.measurement.cpu.field",
    "*.*.uptime.*.host.measurement.field",
    "*.*.load.*.host.measurement..term.field",
    "*.if_octets.*.host.interface.measurement .field.. ",
    "*.if_packets.*.host.interface.measurement. field.. " ,
    "*.if_errors.*.host.interface.measurement .field.. '
    #.app env.service.resource.measurement"
    "servers.*.host.resource.measurement.field*"
    # Default template
    # "server.*"
]

However, I'm getting a few issues with the data that is collected. For example, the `load` series has timestamps that are a factor of 10000 too high (attached screenshot).

I'm unsure what's causing this. Does anyone have any tips? Also open to template suggestions and dashboard JSON recommendations too.

 

[ChrisChros]

Hi does anybody of you know how to backup the database correctly?
I followed the instruction o the InfluxDB Homepage but it looks like its not working.

I want to transfer my database from the jail to my Debian VM.
First I backuped the metastore:

Code:

influxd backup /tmp/backup

then the Database:

Code:

influxd backup -database graphite /tmp/backup

After this was successful I copied the folder /tmp/backup from my jail to the Debian VM, same folder in the root.

Within the Debian VM I executed the command:

Code:

influxd restore -metadir /var/lib/influxdb/meta /tmp/backup

and then restored the Database:

Code:

influxd restore -database graphite -datadir /var/lib/influxdb/data /tmp/backup

finaly I fixed the permissions:

Code:

chown -R influxdb:influxdb /var/lib/influxdb

Everything looks fine but when I start Grafana on the Debian VM and select a Value I cant see any old datapoints. So it loooks like the restore was not really successful.

What did I wrong? any suggestions?

 

[Patrick M. Hausen]

@ChrisChros This is so application specific I guess your chances of getting competent help are higher in the Grafana and/or Influx forum(s).

 

[klewis]

Is there a version of these instructions that are updated for SCALE? I just installed SCALE fresh (Bluefin), and these instructions refence CORE/BSD.

 

[garm]

For the love of all that is holy… do not erase all useful guides by “upgrading” them to SCALE…

 

[klewis]

For the love of all that is holy… do not erase all useful guides by “upgrading” them to SCALE…

I don't think I proposed erasing any guides for CORE, just asking if someone could provide similar guides for SCALE.