FreeNAS daemon is flooding my log with ERR

Status
Not open for further replies.
I

ikorbln

Dabbler
Joined
Apr 13, 2017
Messages
19
Hello @All,

after the last nightly-update my Server is flooding the log with an endless same error.
I dont know what he wants from me......
Code:
Jul 27 11:19:32 freenas daemon[2625]: ==> Log data will now stream in as it occurs:
Jul 27 11:19:32 freenas daemon[2625]:
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:30 [INFO] raft: Restored from snapshot 18-327886-1498904158287
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:32 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:192.168.232.250:8300 Address:192.168.232.250:8300}]
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:32 [INFO] raft: Node at 127.0.0.1:8300 [Follower] entering Follower state (Leader: "")
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:32 [INFO] serf: EventMemberJoin: freenas.fritz.box 127.0.0.1
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:32 [WARN] serf: Failed to re-join any previously known node
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:32 [INFO] consul: Adding LAN server freenas.fritz.box (Addr: tcp/127.0.0.1:8300) (DC: dc1)
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:32 [INFO] serf: EventMemberJoin: freenas.fritz.box.dc1 127.0.0.1
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:32 [INFO] consul: Adding WAN server freenas.fritz.box.dc1 (Addr: tcp/127.0.0.1:8300) (DC: dc1)
Jul 27 11:19:32 freenas daemon[2625]:	 2017/07/27 11:19:32 [WARN] serf: Failed to re-join any previously known node
Jul 27 11:19:37 freenas daemon[2625]:	 2017/07/27 11:19:37 [WARN] raft: not part of stable configuration, aborting election
Jul 27 11:19:39 freenas daemon[2625]:	 2017/07/27 11:19:39 [ERR] agent: failed to sync remote state: No cluster leader
Jul 27 11:19:45 freenas daemon[2625]:	 2017/07/27 11:19:45 [ERR] http: Request GET /v1/kv/consul-alerts/config?dc=dc1&recurse=&token=%22%22, error: No cluster leader from=127.0.0.1:63799
Jul 27 11:19:52 freenas daemon[2625]:	 2017/07/27 11:19:52 [ERR] http: Request GET /v1/health/state/any?dc=dc1&token=%22%22, error: No cluster leader from=127.0.0.1:63803
Jul 27 11:19:57 freenas daemon[2625]: ==> Newer Consul version available: 0.9.0 (currently running: 0.7.1)
Jul 27 11:20:00 freenas daemon[2625]:	 2017/07/27 11:20:00 [ERR] http: Request GET /v1/health/state/any?dc=dc1, error: No cluster leader from=127.0.0.1:63819
Jul 27 11:20:00 freenas daemon[2625]:	 2017/07/27 11:20:00 [ERR] agent: coordinate update error: No cluster leader
Jul 27 11:20:00 freenas daemon[2625]:	 2017/07/27 11:20:00 [ERR] http: Request PUT /v1/session/create?dc=dc1&token=%22%22, error: No cluster leader from=127.0.0.1:63803
Jul 27 11:20:00 freenas daemon[2625]:	 2017/07/27 11:20:00 [ERR] http: Request GET /v1/kv/consul-alerts/config?dc=dc1&recurse=&token=%22%22, error: No cluster leader from=127.0.0.1:63825
Jul 27 11:20:04 freenas daemon[2625]:	 2017/07/27 11:20:04 [ERR] agent: failed to sync remote state: No cluster leader

And then it repeats.....

My Version is: FreeNAS-11-MASTER-201707270433 (7d14267)
 
Last edited by a moderator:
D

dlavigne

Guest
Out of curiosity, why are you running a MASTER rather than the latest release?
 
D

dlavigne

Guest
Unless you originally installed a nightly, at some point you changed to the nightly train. You should reboot and select from the menu a boot environment for stable as you can't "downgrade" from a nightly (you can only boot into a stable).
 
danb35

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,504
ikorbln said:
This is curios, under "check update" it shows me the nightly-train.... And i installed the latest today.....
Click to expand...
So, yes, that's a nightly. Why are you running that rather than a release? IOW, why are you on the nightly train rather than the -STABLE train?
 
I

ikorbln

Dabbler
Joined
Apr 13, 2017
Messages
19
Because it is not a real production system. As Freenas11 came up, it only give a nightly, so i start with nightly.....and the gui say i cant change the train to release.

Code:
You're not allowed to change away from the nightly train, it is considered a downgrade.
If you have an existing boot environment that uses that train, boot into it in order to upgrade that train.


So, should i reinstall?
 
L

level3

Dabbler
Joined
Mar 30, 2017
Messages
12
I also got this issue after upgrade from stable to nightly because of this feature which was not yet released in stable https://bugs.freenas.org/issues/25210 now I also have this message in log every 8 seconds. Is there any way to fix it other than reinstall? Also I didn't get what is the issue with upgrade from stable to nightly? I know that other way around is not supported.
 
M

Mamdoh

Dabbler
Joined
Aug 12, 2017
Messages
12
Update (Aug 14, 2017):

The same issue can be found here as well. I applied a solution suggested here and it seems to have solved the issue.

service consul stop
rm -rf /var/db/system/consul
service consul start


Snippet from the log:

Aug 14 13:56:46 NAS daemon[3017]: 2017/08/14 13:56:46 [ERR] agent: failed to sync remote state: No cluster leader

Aug 14 13:56:56 NAS daemon[3017]: 2017/08/14 13:56:56 [ERR] http: Request PUT /v1/session/create?dc=dc1&token=%22%22, error: No cluster leader from=127.0.0.1:20271

Aug 14 13:56:58 NAS daemon[3017]: ==> Caught signal: terminated

Aug 14 13:56:58 NAS daemon[3017]: 2017/08/14 13:56:58 [INFO] agent: requesting shutdown

Aug 14 13:56:58 NAS daemon[3017]: 2017/08/14 13:56:58 [INFO] consul: shutting down server

Aug 14 13:56:58 NAS daemon[3017]: 2017/08/14 13:56:58 [WARN] serf: Shutdown without a Leave

Aug 14 13:56:58 NAS daemon[3017]: 2017/08/14 13:56:58 [ERR] dns: error starting tcp server: accept tcp 127.0.0.1:8600: use of closed network connection 2017/08/14 13:56:58 [WARN] serf: Shutdown without a Leave

Aug 14 13:56:58 NAS daemon[3017]:

Aug 14 13:56:58 NAS daemon[3017]: 2017/08/14 13:56:58 [INFO] agent: shutdown complete

Aug 14 13:57:33 NAS daemon[27578]: ==> WARNING: BootstrapExpect Mode is specified as 1; this is the same as Bootstrap mode.

Aug 14 13:57:33 NAS daemon[27578]: ==> WARNING: Bootstrap mode enabled! Do not enable unless necessary

Aug 14 13:57:33 NAS daemon[27578]: ==> Starting Consul agent...

Aug 14 13:57:33 NAS daemon[27578]: ==> Starting Consul agent RPC...

Aug 14 13:57:33 NAS daemon[27578]: ==> Consul agent running!

Aug 14 13:57:33 NAS daemon[27578]: Version: 'v0.7.5'

Aug 14 13:57:33 NAS daemon[27578]: Node ID: 'c70f1915-35d5-11e7-834a-xxxx'

Aug 14 13:57:33 NAS daemon[27578]: Node name: 'NAS.local'

Aug 14 13:57:33 NAS daemon[27578]: Datacenter: 'dc1'

Aug 14 13:57:33 NAS daemon[27578]: Server: true (bootstrap: true)

Aug 14 13:57:33 NAS daemon[27578]: Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)

Aug 14 13:57:33 NAS daemon[27578]: Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)

Aug 14 13:57:33 NAS daemon[27578]: Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false

Aug 14 13:57:33 NAS daemon[27578]: Atlas: <disabled>

Aug 14 13:57:33 NAS daemon[27578]:

Aug 14 13:57:33 NAS daemon[27578]: ==> Log data will now stream in as it occurs:

Aug 14 13:57:33 NAS daemon[27578]:

Aug 14 13:57:33 NAS daemon[27578]: 2017/08/14 13:57:33 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:127.0.0.1:8300 Address:127.0.0.1:8300}]

Aug 14 13:57:33 NAS daemon[27578]: 2017/08/14 13:57:33 [INFO] raft: Node at 127.0.0.1:8300 [Follower] entering Follower state (Leader: "")

Aug 14 13:57:33 NAS daemon[27578]: 2017/08/14 13:57:33 [INFO] serf: EventMemberJoin: NAS.local 127.0.0.1

Aug 14 13:57:33 NAS daemon[27578]: 2017/08/14 13:57:33 [INFO] consul: Adding LAN server NAS.local (Addr: tcp/127.0.0.1:8300) (DC: dc1)

Aug 14 13:57:33 NAS daemon[27578]: 2017/08/14 13:57:33 [INFO] serf: EventMemberJoin: NAS.local.dc1 127.0.0.1

Aug 14 13:57:33 NAS daemon[27578]: 2017/08/14 13:57:33 [INFO] consul: Adding WAN server NAS.local.dc1 (Addr: tcp/127.0.0.1:8300) (DC: dc1)

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [WARN] raft: Heartbeat timeout from "" reached, starting election

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [INFO] raft: Node at 127.0.0.1:8300 [Candidate] entering Candidate state in term 2

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [INFO] raft: Election won. Tally: 1

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [INFO] raft: Node at 127.0.0.1:8300 [Leader] entering Leader state

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [INFO] consul: cluster leadership acquired

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [INFO] consul: New leader elected: NAS.local

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [INFO] consul: member 'NAS.local' joined, marking health alive

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [INFO] agent: Synced service 'consul'

Aug 14 13:57:39 NAS daemon[27578]: 2017/08/14 13:57:39 [INFO] agent: Synced service 'nas-health'

Aug 14 13:57:48 NAS daemon[27578]: 2017/08/14 13:57:48 [INFO] agent: Synced check 'service:nas-health'

Aug 14 13:57:59 NAS daemon[27578]: ==> Newer Consul version available: 0.9.2 (currently running: 0.7.1)
Click to expand...

---

Hi all,

I've been facing the same issue/variation of it as well.
Started: A couple of weeks ago.
Current build: FreeNAS-11-MASTER-201708120436.
Machine IP Address: 192.168.2.50
Router IP Address: 192.168.2.1 (also gateway)
Snippet from the log:

Aug 12 16:55:13 NAS root: /etc/rc: WARNING: failed precmd routine for vmware_guestd

Aug 12 16:55:11 NAS smbd: dnssd_clientstub ConnectToServer: connect()-> No of tries: 2

Aug 12 16:55:12 NAS smbd: dnssd_clientstub ConnectToServer: connect()-> No of tries: 3

Aug 12 16:55:13 NAS smbd: dnssd_clientstub ConnectToServer: connect() failed path:/var/run/mdnsd Socket:35 Err:-1 Errno:2 No such file or directory

Aug 12 16:55:21 NAS root: /etc/rc: WARNING: failed precmd routine for minio

Aug 12 16:55:21 NAS netatalk[2936]: Netatalk AFP server starting

Aug 12 16:55:21 NAS netatalk[2936]: Registered with Zeroconf

Aug 12 16:55:21 NAS cnid_metad[2966]: CNID Server listening on localhost:4700

Aug 12 16:55:22 NAS afpd[2965]: Netatalk AFP/TCP listening on 192.168.2.50:548

Aug 12 16:55:23 NAS daemon[3017]: ==> WARNING: BootstrapExpect Mode is specified as 1; this is the same as Bootstrap mode.

Aug 12 16:55:23 NAS daemon[3017]: ==> WARNING: Bootstrap mode enabled! Do not enable unless necessary

Aug 12 16:55:23 NAS daemon[3017]: ==> Starting Consul agent...

Aug 12 16:55:24 NAS daemon[3017]: ==> Starting Consul agent RPC...

Aug 12 16:55:24 NAS daemon[3017]: ==> Consul agent running!

Aug 12 16:55:24 NAS daemon[3017]: Version: 'v0.7.5'

Aug 12 16:55:24 NAS daemon[3017]: Node ID: 'c70f1915-35d5-11e7-834a-xxx’

Aug 12 16:55:24 NAS daemon[3017]: Node name: 'NAS.local'

Aug 12 16:55:24 NAS daemon[3017]: Datacenter: 'dc1'

Aug 12 16:55:24 NAS daemon[3017]: Server: true (bootstrap: true)

Aug 12 16:55:24 NAS daemon[3017]: Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)

Aug 12 16:55:24 NAS daemon[3017]: Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)

Aug 12 16:55:24 NAS daemon[3017]: Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false

Aug 12 16:55:24 NAS daemon[3017]: Atlas: <disabled>

Aug 12 16:55:24 NAS daemon[3017]:

Aug 12 16:55:24 NAS daemon[3017]: ==> Log data will now stream in as it occurs:

Aug 12 16:55:24 NAS daemon[3017]:

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:23 [INFO] raft: Restored from snapshot 2-24589-1494915727620

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:24 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:192.168.2.3:8300 Address:192.168.2.3:8300}]

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:24 [INFO] raft: Node at 127.0.0.1:8300 [Follower] entering Follower state (Leader: "")

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:24 [INFO] serf: EventMemberJoin: NAS.local 127.0.0.1

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:24 [INFO] serf: Attempting re-join to previously known node: freenas.local: 192.168.2.3:8301

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:24 [INFO] consul: Adding LAN server NAS.local (Addr: tcp/127.0.0.1:8300) (DC: dc1)

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:24 [INFO] serf: EventMemberJoin: NAS.local.dc1 127.0.0.1

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:24 [INFO] consul: Adding WAN server NAS.local.dc1 (Addr: tcp/127.0.0.1:8300) (DC: dc1)

Aug 12 16:55:24 NAS daemon[3017]: 2017/08/12 16:55:24 [INFO] serf: Attempting re-join to previously known node: freenas.local.dc1: 192.168.2.3:8302

Aug 12 16:55:31 NAS daemon[3017]: 2017/08/12 16:55:31 [ERR] agent: failed to sync remote state: No cluster leader

Aug 12 16:55:33 NAS daemon[3017]: 2017/08/12 16:55:33 [WARN] raft: not part of stable configuration, aborting election

Aug 12 16:55:34 NAS daemon[3017]: 2017/08/12 16:55:34 [WARN] serf: Failed to re-join any previously known node

Aug 12 16:55:40 NAS daemon[3017]: 2017/08/12 16:55:40 [WARN] serf: Failed to re-join any previously known node

Aug 12 16:55:40 NAS daemon[3017]: 2017/08/12 16:55:40 [ERR] http: Request GET /v1/kv/consul-alerts/config?dc=dc1&recurse=&token=%22%22, error: No cluster leader from=127.0.0.1:20266

Aug 12 16:55:47 NAS daemon[3017]: ==> Newer Consul version available: 0.9.2 (currently running: 0.7.1)

Aug 12 16:55:47 NAS daemon[3017]: 2017/08/12 16:55:47 [ERR] http: Request GET /v1/health/state/any?dc=dc1&token=%22%22, error: No cluster leader from=127.0.0.1:20271

Aug 12 16:55:54 NAS daemon[3017]: 2017/08/12 16:55:54 [ERR] agent: coordinate update error: No cluster leader

Aug 12 16:55:54 NAS daemon[3017]: 2017/08/12 16:55:54 [ERR] http: Request GET /v1/kv/consul-alerts/config?dc=dc1&recurse=&token=%22%22, error: No cluster leader from=127.0.0.1:20295

Aug 12 16:55:55 NAS daemon[3017]: 2017/08/12 16:55:55 [ERR] http: Request GET /v1/health/state/any?dc=dc1, error: No cluster leader from=127.0.0.1:20289

Aug 12 16:55:55 NAS daemon[3017]: 2017/08/12 16:55:55 [ERR] http: Request PUT /v1/session/create?dc=dc1&token=%22%22, error: No cluster leader from=127.0.0.1:20271

Aug 12 16:55:56 NAS daemon[3017]: 2017/08/12 16:55:56 [ERR] agent: failed to sync remote state: No cluster leader

Aug 12 16:56:07 NAS daemon[3017]: 2017/08/12 16:56:07 [ERR] http: Request GET /v1/health/state/any?dc=dc1, error: No cluster leader from=127.0.0.1:20289

Aug 12 16:56:12 NAS daemon[3017]: 2017/08/12 16:56:12 [ERR] http: Request PUT /v1/session/create?dc=dc1&token=%22%22, error: No cluster leader from=127.0.0.1:20271

Aug 12 16:56:21 NAS daemon[3017]: 2017/08/12 16:56:21 [ERR] agent: failed to sync remote state: No cluster leader

Aug 12 16:56:28 NAS daemon[3017]: 2017/08/12 16:56:28 [ERR] agent: coordinate update error: No cluster leader

Aug 12 16:56:29 NAS daemon[3017]: 2017/08/12 16:56:29 [ERR] http: Request PUT /v1/session/create?dc=dc1&token=%22%22, error: No cluster leader from=127.0.0.1:20271

Aug 12 16:56:34 NAS daemon[3017]: 2017/08/12 16:56:34 [ERR] http: Request GET /v1/health/state/any?dc=dc1, error: No cluster leader from=127.0.0.1:20289
Click to expand...

… Last few messages keep repeating every few seconds.

Thank you for your help in advance.

Mamdoh
 
Last edited:
Status
Not open for further replies.

Similar threads

R
  • Locked
Upgraded to 11.1-RC1 and still getting Overheating errors
Replies
5
Views
2K
Ray Milyard
R
D
  • Locked
freenas.local daily security run output - login failures
Replies
10
Views
5K
Chris Moore
Chris Moore
Hugo Ochoa
  • Locked
[SOLVED] Time Machine doesn't find AFP share. Help please!!!
Replies
4
Views
5K
Hugo Ochoa
Hugo Ochoa
P
  • Locked
Home user - should I upgrade my ancient version (9.3)?
Replies
4
Views
1K
danb35
danb35
S
  • Locked
FreeNAS components crashing
2
Replies
26
Views
6K
DrKK
DrKK
Top