Kubernetes install failure (Get "http://localhost:10248/healthz": dial tcp 127.0.0.1:10248: connect: connection refused.)

General Discussions
1

Cluster information: I’m deploying kubernetes cluster master and node when. It failed during the installation. Below is the output.

[kubelet-check] Initial timeout of 40s passed.
[kubelet-check] It seems like the kubelet isn’t running or healthy.
[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz’ failed with error: Get “http://localhost:10248/healthz”: dial tcp 127.0.0.1:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn’t running or healthy.
[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz’ failed with error: Get “http://localhost:10248/healthz”: dial tcp 127.0.0.1:10248: connect: connection refused.

I also see the kubelet service is in restart loop.

2

kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: activating (auto-restart) (Result: exit-code) since Wed 2022-01-19 16:13:24 UTC; 6s ago
Docs: Kubernetes Documentation | Kubernetes
Process: 27211 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=
Main PID: 27211 (code=exited, status=1/FAILURE)
lines 1-8/8 (END)

3

a couple of question before we start searching for a solution:

  1. could you please provide the kubernetes version you are installing?
  2. could you please provide the OS version and basic conf you are using?
  3. could you please provide the last lets say 30 lines from this command (while restarting kubelet):
journalctl -u kubelet -f
1 Like
4

Hi
following this document How to Install a Kubernetes Cluster on CentOS 8
[root@master ~]# kubeadm version
kubeadm version: &version.Info{Major:“1”, Minor:“23”, GitVersion:“v1.23.5”, GitCommit:“c285e781331a3785a7f436042c65c5641ce8a9e9”, GitTreeState:“clean”, BuildDate:“2022-03-16T15:57:37Z”, GoVersion:“go1.17.8”, Compiler:“gc”, Platform:“linux/amd64”}

Apr 05 23:29:44 master.giri.com kubelet[253558]: E0405 23:29:44.252560 253558 server.go:302] “Failed to run kubelet” err=“failed to run Kubelet: misconfiguration: kubelet cgroup driver: “systemd” is different from docker cgroup driver: “cgroupfs””
Apr 05 23:29:44 master.giri.com systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Apr 05 23:29:44 master.giri.com systemd[1]: kubelet.service: Failed with result ‘exit-code’.

5

Kubernetes version: v1.24.0
Ubuntu 18.04

[kubelet-check] It seems like the kubelet isn’t running or healthy.
[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz’ failed with error: Get “http://localhost:10248/healthz”: dial tcp 127.0.0.1:10248: connect: connection

6

If u have followed installation guide by K8s except docker installation then this might help.
in my case, path of docker installation is causes the same issue .
issue is kubelet that is exited with status 1
OS-ubuntu 20.04

1 . install docker through k8s guide (command - apt install docker.io does not install latest
version of docker and there can be issue in installation of containerd as well)
Install Docker Engine on Ubuntu | Docker Documentation – it install latest version of docker as well as containerd.
2. after installtion there will be /etc/containerd/config.toml file by default . just delete it
3. systemctl restart containerd
4. systemctl restart kubelet