Unlocking an encrypted pool - collectd fails to start

[FlyingBear]

Here's an odd one: I have the majority of my data on an unencrypted pool. I also have a small (2x8TB SAS mirror) pool for secure storage. I unlock it when I want to back up secure data. Almost (but not every) time that I unlock that pool using the GUI, collectd fails to restart.

Upon inspection, the collectd.conf file is corrupt: after line 115, 20 or so lines from earlier in the file are appended. When I edit them out, I can restart collectd.conf. Of course, the file comes back to its senses on its own on a reboot.

Data corruption is scary, but the behavior is consistent when the problem occurs, so I tend to think that it's a bug versus some memory/disk error. Any ideas please?

 

[dlavigne]

Sounds like something that should be reported to bugs.freenas.org. Please post the issue number here if you do.

 

[FlyingBear]

Thanks, will do. Of course, it won't do it now. I don't want to reboot the server right now -- it's busy -- but I suspect, based on the pattern of problems, that it only happens on the first unlock after a boot. I'll do some tests this weekend when the server isn't cranking.

 

[trinxie]

I have exactly this problem
My report graphs were missing after installing 11.2. Cannot restart collectd
I am not sure how to edit the file..
Is your problem gone after editing the collectd.conf file?


Error when starting collectd:
root@nas:/var/db/collectd/rrd/localhost # /usr/local/etc/rc.d/collectd start
Starting collectd.
Parse error in file `/usr/local/etc/collectd.conf', line 116 near `/': syntax error, unexpected SLASH, expecting UNQUOTED_STRING
yyparse returned error #1
configfile: Cannot read file `/usr/local/etc/collectd.conf'.
Unable to read config file /usr/local/etc/collectd.conf.
Error: Reading the config file failed!
Read the logs for details.
/usr/local/etc/rc.d/collectd: WARNING: failed to start collectd
root@nas:/var/db/collectd/rrd/localhost #

 

[trinxie]

I consider this solved for now. Have not re-booted yet....

This is how my file looks like at row 116

Removed all after #113:
#114 is end of file.

Collectd started:
root@nas:/usr/local/etc # /usr/local/etc/rc.d/collectd start
Starting collectd.
plugin_load: plugin "aggregation" successfully loaded.
plugin_load: plugin "cpu" successfully loaded.
plugin_load: plugin "cputemp" successfully loaded.
plugin_load: plugin "ctl" successfully loaded.
plugin_load: plugin "df" successfully loaded.
plugin_load: plugin "disk" successfully loaded.
plugin_load: plugin "exec" successfully loaded.
plugin_load: plugin "geom_stat" successfully loaded.
plugin_load: plugin "interface" successfully loaded.
plugin_load: plugin "load" successfully loaded.
plugin_load: plugin "memory" successfully loaded.
plugin_load: plugin "network" successfully loaded.
plugin_load: plugin "processes" successfully loaded.
plugin_load: plugin "python" successfully loaded.
plugin_load: plugin "rrdtool" successfully loaded.
plugin_load: plugin "swap" successfully loaded.
plugin_load: plugin "uptime" successfully loaded.
root@nas:/usr/local/etc #

Now I get reports except for the CPU, but that seems to be a known issue.

 

[FlyingBear]

I have exactly this problem
My report graphs were missing after installing 11.2. Cannot restart collectd
I am not sure how to edit the file..
Is your problem gone after editing the collectd.conf file?


Error when starting collectd:
root@nas:/var/db/collectd/rrd/localhost # /usr/local/etc/rc.d/collectd start
Starting collectd.
Parse error in file `/usr/local/etc/collectd.conf', line 116 near `/': syntax error, unexpected SLASH, expecting UNQUOTED_STRING
yyparse returned error #1
configfile: Cannot read file `/usr/local/etc/collectd.conf'.
Unable to read config file /usr/local/etc/collectd.conf.
Error: Reading the config file failed!
Read the logs for details.
/usr/local/etc/rc.d/collectd: WARNING: failed to start collectd
root@nas:/var/db/collectd/rrd/localhost #

That's precisely the same problem that I have. Editing the file and restarting collectd solves the problem. I guess I should take a copy of the correct file and use a script to copy it back and restart collectd, but i haven't rebooted my FreeNAS in weeks.

 

[trinxie]

That is a good idea. I have made a copy and will re-boot tomorrow. I will post the result !

 

[FlyingBear]

I just installed 11.2-STABLE and the problem seems to have gone away. I can't be sure, but I think that the problem may have arisen when using the new UI and unlocking an encrypted volume, versus using the old UI. This is a hunch, not a fact.

 

[trinxie]

I noticed one thing today
I locked my Volume containing the System Dataset. Reporting database is set to System Dataset

Then I got these errors in the /var/log/messages:

Code:

Dec 12 18:50:05 nas collectd[73579]: check_create_dir: `/var/db/collectd/rrd' exists but is not a directory!
Dec 12 18:50:05 nas collectd[73579]: check_create_dir: `/var/db/collectd/rrd' exists but is not a directory!
Dec 12 18:50:05 nas collectd[73579]: check_create_dir: `/var/db/collectd/rrd' exists but is not a directory!

It seems like /usr/local/etc/collectd.conf got corrupted again.

Tried to restart collectd again:

Code:

root@nas:/var/db/collectd # /usr/local/etc/rc.d/collectd start
Starting collectd.
Parse error in file `/usr/local/etc/collectd.conf', line 116 near `/': syntax error, unexpected SLASH, expecting UNQUOTED_STRING yyparse returned error #1
configfile: Cannot read file `/usr/local/etc/collectd.conf'.
Unable to read config file /usr/local/etc/collectd.conf.
Error: Reading the config file failed!
Read the logs for details.
/usr/local/etc/rc.d/collectd: WARNING: failed to start collectd

Is it wrong having the reporting database set to System Dataset when pool is encrypted?