Unable to run alert source 'HasUpdate‘

bigdadda06

bigdadda06

Dabbler
Joined
Aug 23, 2018
Messages
14
So today I’ve been getting a few of these alerts. Anyone know what they mean?
The boot pool is currently doing a resilver? Could the be the cause?

Running 11.2 on a i3 with 8GB RAM, it was running fine with no alerts until I added another USB drive to the boot pool, so that is probably it.
It seems to be taking a long time to resolver the 16GB drive however.

Code:
New alerts:
* Unable to run alert source 'HasUpdate'
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 358, in __run_source
   alerts = (await alert_source.check()) or []
File "/usr/local/lib/python3.6/site-packages/middlewared/alert/base.py", line 96, in check
   return await self.middleware.run_in_thread(self.check_sync)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1009, in run_in_thread
   return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
   result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/source/update.py", line 25, in check_sync
   self.middleware.call_sync("datastore.query", "system.update", None, {"get": True})
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1139, in call_sync
   return fut.result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 425, in result
   return self.__get_result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
   raise self._exception
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1063, in _call
   return await run_method(methodobj, *args)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1009, in run_in_thread
   return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
   result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
   return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 175, in query
   select=options.get('select'),
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 88, in __queryset_serialize
   for i in qs:
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __iter__
   self._fetch_all()
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1087, in _fetch_all
   self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 54, in __iter__
   results = compiler.execute_sql()
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
   cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
   six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
   raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
django.db.utils.OperationalError: database is locked


Alerts:
* Unable to run alert source 'HasUpdate'
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/alert.py", line 358, in __run_source
   alerts = (await alert_source.check()) or []
File "/usr/local/lib/python3.6/site-packages/middlewared/alert/base.py", line 96, in check
   return await self.middleware.run_in_thread(self.check_sync)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1009, in run_in_thread
   return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
   result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/../alert/source/update.py", line 25, in check_sync
   self.middleware.call_sync("datastore.query", "system.update", None, {"get": True})
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1139, in call_sync
   return fut.result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 425, in result
   return self.__get_result()
File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
   raise self._exception
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1063, in _call
   return await run_method(methodobj, *args)
File "/usr/local/lib/python3.6/site-packages/middlewared/main.py", line 1009, in run_in_thread
   return await self.loop.run_in_executor(executor, functools.partial(method, *args, **kwargs))
File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
   result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/schema.py", line 668, in nf
   return f(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 175, in query
   select=options.get('select'),
File "/usr/local/lib/python3.6/site-packages/middlewared/plugins/datastore.py", line 88, in __queryset_serialize
   for i in qs:
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 256, in __iter__
   self._fetch_all()
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 1087, in _fetch_all
   self._result_cache = list(self.iterator())
File "/usr/local/lib/python3.6/site-packages/django/db/models/query.py", line 54, in __iter__
   results = compiler.execute_sql()
File "/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
   cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 94, in __exit__
   six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.6/site-packages/django/utils/six.py", line 685, in reraise
   raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 64, in execute
   return self.cursor.execute(sql, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 412, in execute
   execute = self.locked_retry(Database.Cursor.execute, query, params)
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 403, in locked_retry
   raise e
File "/usr/local/www/freenasUI/freeadmin/sqlite3_ha/base.py", line 389, in locked_retry
   rv = method(self, *args, **kwargs)
django.db.utils.OperationalError: database is locked
 
Last edited:
I

Inxsible

Guru
Joined
Aug 14, 2017
Messages
1,123
I have similar issues over last month as well.

I have seen alerts related to 'HasUpdate', 'Samba4', 'ZPoolCapacity', 'smartd'

The Samba4 and smartd alerts --- I get a "Gone alert" email within 1 minute -- indicating that the alert has been resolved.
ZPoolCapacity -- I get a "Gone alert" email in 5 minutes
HasUpdate -- I get a "Gone alert" email in 1 hour


What is causing these alerts? How do they fix themselves?

How can I stop having these alerts--- Do I have to reschedule the timings of my scrubs etc?
 
Chris Moore

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Please put long text listings like this inside code tags like this:

1547876512577.png


that way they come out looking much more readable.
 
bigdadda06

bigdadda06

Dabbler
Joined
Aug 23, 2018
Messages
14
Inxsible said:
I have similar issues over last month as well.
I have seen alerts related to 'HasUpdate', 'Samba4', 'ZPoolCapacity', 'smartd'
The Samba4 and smartd alerts --- I get a "Gone alert" email within 1 minute -- indicating that the alert has been resolved.
ZPoolCapacity -- I get a "Gone alert" email in 5 minutes
HasUpdate -- I get a "Gone alert" email in 1 hour

What is causing these alerts? How do they fix themselves?
How can I stop having these alerts--- Do I have to reschedule the timings of my scrubs etc?
Click to expand...

I think I have seen a couple of different versions as well. i.e. for HasUpdate as well as Replication, and I usually get a "Alert Gone" message as well.

My resolver has stopped now, but I think I'm still getting the messages.
 
bigdadda06

bigdadda06

Dabbler
Joined
Aug 23, 2018
Messages
14
dlavigne said:
It looks like the most likely culprit is slow USB boot devices: https://redmine.ixsystems.com/issues/69159. U2 will add a timeout to help minimize those alerts, a better solution is replace slow boot devices with better ones.
Click to expand...

That appears to be correct, I removed the "new" USB stick and the alerts appear to have stopped.
I say "new" becuase I was using the same USB stick with 11.1 U6.3 without error.
Or maybe I should say without error that it was telling me about??? :smile:
 
I

Inxsible

Guru
Joined
Aug 14, 2017
Messages
1,123
That's interesting. I didn't change the config of my setup at all and upgrading seems to have caused the start of these Alerts.

Unfortunately, in this rig, I don't have any spare sata ports to get rid of my USB :( So I might have to just live with it until the upgrade to U2 -- as @dlavigne suggests
 
You must log in or register to reply here.

Similar threads

TAC
  • Locked
Gone alerts: * Unable to run alert source 'Replication'
Replies
1
Views
977
Ericloewe
Ericloewe
hiro5id
My FreeNAS-11.2-RELEASE-U1 appears to be working fine. However i'm getting these errors emailed to me.
Replies
3
Views
1K
hiro5id
hiro5id
Z
smartd Issue
Replies
8
Views
1K
dlavigne
D
jacob42
HTTPDBindAddress errors - likely something else under the hood
Replies
1
Views
1K
dlavigne
D
J
New Alerts and gone alerts
Replies
6
Views
1K
mistermanko
mistermanko
Top