Hi,
I am unable to get k3s service to start. The system had an uptime of over a year beforehand, but was having trouble recently in updating one of the apps, so I rebooted the system and then got hit with the "Application are not running" screen when i look. Unable to install new ones either. Updated SCALE to the latest, and that didn't work. Have been trying all the various solutions on here:
- unset, set pool and restart - didnt work
- set to new pool and set back - didnt work (new pool did seem to work on its own though)
Below is the status output for k3s which doesn't match and other peoples issues. Hopefully someone has a thought?
I am unable to get k3s service to start. The system had an uptime of over a year beforehand, but was having trouble recently in updating one of the apps, so I rebooted the system and then got hit with the "Application are not running" screen when i look. Unable to install new ones either. Updated SCALE to the latest, and that didn't work. Have been trying all the various solutions on here:
- unset, set pool and restart - didnt work
- set to new pool and set back - didnt work (new pool did seem to work on its own though)
Below is the status output for k3s which doesn't match and other peoples issues. Hopefully someone has a thought?
Code:
root@truenas[~]# systemctl status k3s ● k3s.service - Lightweight Kubernetes Loaded: loaded (/lib/systemd/system/k3s.service; disabled; preset: disabled) Active: activating (start) since Sun 2023-07-30 21:17:18 BST; 45s ago Docs: https://k3s.io Process: 1226519 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS) Process: 1226520 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS) Main PID: 1226521 (k3s-server) Tasks: 104 Memory: 625.6M CPU: 2min 8.814s CGroup: /system.slice/k3s.service └─1226521 "/usr/local/bin/k3s server" Jul 30 21:17:59 truenas k3s[1226521]: E0730 21:17:59.210574 1226521 storage_rbac.go:232] unable to reconcile clusterrole.rbac.authorization.k8s.io/system:controller:statefulset-controller: exceeded maximum attempts Jul 30 21:17:59 truenas k3s[1226521]: E0730 21:17:59.807876 1226521 storage_rbac.go:232] unable to reconcile clusterrole.rbac.authorization.k8s.io/system:controller:ttl-controller: exceeded maximum attempts Jul 30 21:17:59 truenas k3s[1226521]: time="2023-07-30T21:17:59+01:00" level=info msg="Tunnel server egress proxy waiting for runtime core to become available" Jul 30 21:18:00 truenas k3s[1226521]: time="2023-07-30T21:18:00+01:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:6443/v1-k3s/readyz: 500 Internal Server Error" Jul 30 21:18:00 truenas k3s[1226521]: E0730 21:18:00.402094 1226521 storage_rbac.go:232] unable to reconcile clusterrole.rbac.authorization.k8s.io/system:controller:certificate-controller: exceeded maximum attempts Jul 30 21:18:01 truenas k3s[1226521]: E0730 21:18:01.011501 1226521 storage_rbac.go:232] unable to reconcile clusterrole.rbac.authorization.k8s.io/system:controller:pvc-protection-controller: exceeded maximum attempts Jul 30 21:18:01 truenas k3s[1226521]: E0730 21:18:01.611309 1226521 storage_rbac.go:232] unable to reconcile clusterrole.rbac.authorization.k8s.io/system:controller:pv-protection-controller: exceeded maximum attempts Jul 30 21:18:02 truenas k3s[1226521]: E0730 21:18:02.212712 1226521 storage_rbac.go:232] unable to reconcile clusterrole.rbac.authorization.k8s.io/system:controller:ttl-after-finished-controller: exceeded maximum attempts Jul 30 21:18:02 truenas k3s[1226521]: E0730 21:18:02.811431 1226521 storage_rbac.go:232] unable to reconcile clusterrole.rbac.authorization.k8s.io/system:controller:root-ca-cert-publisher: exceeded maximum attempts Jul 30 21:18:03 truenas k3s[1226521]: E0730 21:18:03.614929 1226521 storage_rbac.go:264] unable to reconcile clusterrolebinding.rbac.authorization.k8s.io/cluster-admin: exceeded maximum attempts