Log shows "[reporting.rrd:163] Failed to generate graph"

[-fun-]

I moved my pool from a single disk to a mirror recently and since then I get numerous messages as shown in the subject line in my log. This probably corresponds to 2 graphs not being displayed in the Reporting / Network section.

My two regular NICs and a bridge0 and tap0 are shown fine, another 2 graphs are missing like this:

ifconfig tells me I have vnet0:1 and vnet0:2 interfaces in addition to the ones showing fine in the reporting.

Any advice what could be done to correct the problem?

 

[Chris Moore]

You could try moving the system dataset.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Chris Moore]

Any advice what could be done to correct the problem?

The 'system dataset' is where the information is stored. If you move the system dataset, it is supposed to create the new storage location and start writing the data there. Where is your system dataset stored now?

 

[-fun-]

I moved the system dataset to the boot volume (2 mirrored USB flash drives) and disabled the RRD data. After moving the pool I put the system dataset back to the main pool, enabling RRD data again. All historic RRD data was even preserved after this. So by putting the system dataset to the main pool a new storage location was not created but the existing location was reused including existing data.

Would it help to destroy the historic data and clear the storage location? If so, how should I do this?

 

[-fun-]

Some more analysis: This broken graphs have these URLs:

http://unger.funet.me/reporting/gen...step=0&identifier=vnet0:1&cache=1518157150803

http://unger.funet.me/reporting/gen...step=0&identifier=vnet0:2&cache=1518157150806

These obviously refer to the vnet0:1 and vnet0:2 interfaces. These interfaces are associated with 2 iocage jails which are running fine.

 

[-fun-]

Same thing with new UI, just there are 3 graphs per interface not showing and many more error messages in the log ... :-(

Deleting /var/db/collectd/rrd/localhost/interface* didn't change anything, so the trouble is not caused by corrupted files. Are the graphs created form the files or from data in memory?

 

[-fun-]

I'm slowly getting somewhere. The error message

Feb 10 09:48:28 unger uwsgi: [reporting.rrd:163] Failed to generate graph: b'ERROR: bad CF: rx\n'

is clearly created by rrdtool. rrdtool requires colons as limiters of the command line. Syntax for this:

DEF:<vname>=<rrdfile>:<ds-name>:<CF>[:step=<step>][:start=<time>][:end=<time>][:reduce=<CF>][:daemon=<address>]

Apparently the "rx\n" is considered part of the CF statement and provokes the error message provided above. CF ("consolidation function") can be any of AVERAGE, MINIMUM, MAXIMUM, and LAST.

So I strongly suspect that the colon from the interface id is not escaped correctly and the "rx" part is meant to provide part of the interface name in some strange way. To me this smells like an error in FreeNAS.

Has anyone here network interface rrd graphs displayed correctly for interfaces with colons as part of their names like vnet0:3?

 

[georgep56]

I came across this post while googling the problem.
I have the same error - also with jails using network interfaces containing a colon in the name.

Code:

root@freenas:~ # ifconfig vnet0:1
vnet0:1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
   description: associated with jail: Nextcloud1
   options=8<VLAN_MTU>
   ether 02:ff:60:1c:76:6a
   hwaddr 02:45:90:00:04:0a
   nd6 options=1<PERFORMNUD>
   media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
   status: active
   groups: epair

Code:

root@freenas:/var/log # tail -7 messages
Feb 22 13:45:07 freenas kernel: epair3a: promiscuous mode enabled
Feb 22 13:56:56 freenas kernel: ifa_maintain_loopback_route: deletion failed for interface lo0: 48
Feb 22 15:01:56 freenas uwsgi: [reporting.rrd:163] Failed to generate graph: b'ERROR: bad CF: rx\n'
Feb 22 15:02:46 freenas uwsgi: [reporting.rrd:163] Failed to generate graph: b'ERROR: bad CF: rx\n'
Feb 22 17:30:57 freenas uwsgi: [reporting.rrd:163] Failed to generate graph: b'ERROR: bad CF: rx\n'
Feb 22 17:35:58 freenas uwsgi: [reporting.rrd:163] Failed to generate graph: b'ERROR: bad CF: rx\n'
Feb 22 18:01:51 freenas uwsgi: [reporting.rrd:163] Failed to generate graph: b'ERROR: bad CF: rx\n'
root@freenas:/var/log #

Can I provide any more information to help with diagnosis?

 

[Chris Moore]

I think you should file a bug report.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[georgep56]

Looks like it's been addressed already

https://redmine.ixsystems.com/issues/28470