plugin_dispatch_values: Low water mark reached. Dropping 100% of metrics

[Paul042020]

Hello and happy new year to all.
This morning I had a little problem.
No more access to nas (data and administration interface).
Fortunately, I was able to connect to ipmi.
On the console I could see the following message:
plugin_dispatch_values: Low water mark reached. Dropping 100% of metrics

I restarted the server and everything went back to normal.

Should I be worried?
Where is this coming from?

Sincerely

 

[Darknight_DS]

I'm having the exact same issue. It also has been causing my boot disks to not wake up properly. I just set my boot disks yesterday to always on. I see this issue has been around for a few years. No one has pin pointed the cause yet. BTW my system crashed again today with this error.

 

[jomom]

been experiencing the same issue this week, already replaced all drives in my main pool. Deleted and recreated it as a new pool, but nothings helped yet.

Seems to only happen once there is a bunch of data on the pool.

i'm going to attempt to remove the cache drive from the pool and use that for the plugins and see if that helps

 

[Darknight_DS]

So, is this some sort of zfs replication issue? I removed one of the drives in my boot pool and the issues stopped.

 

[jomom]

my current running theory is that my drive enclosure isn't getting enough amps from the power adapters I have.
Before all this started I replaced one drive with a higher wattage one and it might have tipped the scales. So all my troubleshooting after that was moot.
I have frankenstein'd an old 300w power supply to test with and am going to run through some tests in the next couple of days to be sure

 

[Darknight_DS]

Spoke too soon, everything was good since my last post until about 3 days ago and now my system has crashed with the same collected low watermark error. Does anyone know why this is happening and how to resolve this issue?

 

[jomom]

Sorry, I must have forgotten to post a follow up. For me it was definitely the power supply not allowing the drives to spin up fully when they needed to, presumably when a lot of reads/writes were happening, causing corruption in the pool

 

[Paul042020]

Hello,

It's amazing that so many of us have this problem.
Have you made any progress with your research?
As far as I'm concerned, not really. When it happens, I restart the nas.

Here's a screenshot of the message that appears on the console when I connect in ipmi :

Do you have the same message?

The strangest thing is that there are no specific Truenas or sensor alerts visible on the ipmi interface. Everything is "normal".

Regards

 

[dejandro]

In case it might help you, as it works in my case, after disabling the S.M.A.R.T. service and unchecking the box to Start Automatically, the system is stable, with no interruptions. This may be a workaround but shouldn't be recommended for long-term action.

 

[Paul042020]

Hello,

I'm here with some news.
I think I've solved my problem.
I changed my email address 6 months ago, but forgot to edit and set up the new email in Truenas.
So Truenas could no longer send emails.
I set up the correct email. I haven't had this error message since March 13.
For the moment, I'm keeping an eye out to see if this is the cause of the problem.

I don't know why a wrong e-mail account would cause truenas to bug, but if anyone has an explanation, I'll let you know.

Regards