# Kube-apiserver is gone / The connection to the server master:6443 was refused - did you specify the right host or port? **URL:** \ **Category:** General Discussions\ **Created:** [January 17, 2024, 9:06pm UTC](https://discuss.kubernetes.io/t/kube-apiserver-is-gone-the-connection-to-the-server-master-6443-was-refused-did-you-specify-the-right-host-or-port/26754 "2024-01-17T21:06:32Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![czezz](https://avatars.discourse-cdn.com/v4/letter/c/54ee81/32.png) [@czezz](https://discuss.kubernetes.io/u/czezz)\ **Post date:** [January 17, 2024, 9:06pm UTC](https://discuss.kubernetes.io/t/kube-apiserver-is-gone-the-connection-to-the-server-master-6443-was-refused-did-you-specify-the-right-host-or-port/26754/1 "2024-01-17T21:06:32Z")
### Cluster information: Kubernetes version: 1.20.1 Cloud being used: Virtualbox Installation method: [Build your own Multi-node Kubernetes Cluster with Ubuntu(20) on VirtualBox | techbeatly](https://www.techbeatly.com/build-your-own-multi-node-kubernetes-cluster-with-ubuntu20-on-virtualbox/) Host OS: Ubuntu 20.04.4 LTS CNI and version: calico CRI and version: This is my virtualbox kubernetes lab which I didnt use for a few months. I have started it today and any kubectl command ends up like this: ```auto $ kubectl get nodes -o wide The connection to the server master:6443 was refused - did you specify the right host or port? $ kubectl get po --all-namespaces The connection to the server master:6443 was refused - did you specify the right host or port? ``` Indeed, there is nothing listening on socket master:6443 ```auto $ ss -tl4np State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 4096 127.0.0.1:34627 0.0.0.0:* LISTEN 0 4096 127.0.0.1:10248 0.0.0.0:* LISTEN 0 4096 192.168.1.190:2379 0.0.0.0:* LISTEN 0 4096 127.0.0.1:2379 0.0.0.0:* LISTEN 0 4096 192.168.1.190:2380 0.0.0.0:* LISTEN 0 4096 127.0.0.1:2381 0.0.0.0:* LISTEN 0 4096 127.0.0.1:33133 0.0.0.0:* LISTEN 0 4096 127.0.0.1:10257 0.0.0.0:* LISTEN 0 4096 127.0.0.1:10259 0.0.0.0:* LISTEN 0 4096 127.0.0.53%lo:53 0.0.0.0:* LISTEN 0 128 0.0.0.0:22 0.0.0.0:* ccd@master:~$ curl https://master:6443 curl: (7) Failed to connect to master port 6443: Connection refused ``` As what I can see on the other cluster, process responsible for service on port 6443 is **kube-apiserver**. There is no such binary. **Question:** 1. How do I start kube-apiserver 2. Why all of the suddent kube-apiserver is gone. It was working quite good last time i was this lab. Side note: kubelet service is running but in in error state. ```auto systemctl status kubelet ? kubelet.service - kubelet: The Kubernetes Node Agent Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/kubelet.service.d +-10-kubeadm.conf Active: active (running) since Wed 2024-01-17 19:55:47 UTC; 44min ago Docs: https://kubernetes.io/docs/home/ Main PID: 613 (kubelet) Tasks: 15 (limit: 2274) Memory: 128.5M CGroup: /system.slice/kubelet.service +-613 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.6 Jan 17 20:40:43 master kubelet[613]: E0117 20:40:43.451465 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:43 master kubelet[613]: E0117 20:40:43.552175 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:43 master kubelet[613]: E0117 20:40:43.653822 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:43 master kubelet[613]: E0117 20:40:43.754428 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:43 master kubelet[613]: E0117 20:40:43.855082 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:43 master kubelet[613]: E0117 20:40:43.957374 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:44 master kubelet[613]: E0117 20:40:44.058207 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:44 master kubelet[613]: E0117 20:40:44.158993 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:44 master kubelet[613]: E0117 20:40:44.259665 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" Jan 17 20:40:44 master kubelet[613]: E0117 20:40:44.360617 613 kubelet.go:2422] "Error getting node" err="node \"master\" not found" ``` ---
**Author:** ![czezz](https://avatars.discourse-cdn.com/v4/letter/c/54ee81/32.png) [@czezz](https://discuss.kubernetes.io/u/czezz)\ **Post date:** [January 18, 2024, 5:19pm UTC](https://discuss.kubernetes.io/t/kube-apiserver-is-gone-the-connection-to-the-server-master-6443-was-refused-did-you-specify-the-right-host-or-port/26754/2 "2024-01-18T17:19:23Z")
**[UPDATE]** It seems that in my lab kube-apiserver is running as docker container: ```auto root@master:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0adc676e6077 595f327f224a "kube-scheduler --au…" 10 minutes ago Up 10 minutes k8s_kube-scheduler_kube-scheduler-master_kube-system_172567758300a3c99b36d0d4efd9321a_7 8e5200bd131f 25f8c7f3da61 "etcd --advertise-cl…" 10 minutes ago Up 10 minutes k8s_etcd_etcd-master_kube-system_dd8c7aeab8fa24c31e933da39dcafd96_7 7ad4134212ce df7b72818ad2 "kube-controller-man…" 10 minutes ago Up 10 minutes k8s_kube-controller-manager_kube-controller-manager-master_kube-system_b38aa758e725e1c490cee42d42ec8bff_7 6baca0741b74 k8s.gcr.io/pause:3.6 "/pause" 10 minutes ago Up 10 minutes k8s_POD_etcd-master_kube-system_dd8c7aeab8fa24c31e933da39dcafd96_7 9a5572813b8a k8s.gcr.io/pause:3.6 "/pause" 10 minutes ago Up 10 minutes k8s_POD_kube-controller-manager-master_kube-system_b38aa758e725e1c490cee42d42ec8bff_7 daca7dae837c k8s.gcr.io/pause:3.6 "/pause" 10 minutes ago Up 10 minutes k8s_POD_kube-scheduler-master_kube-system_172567758300a3c99b36d0d4efd9321a_7 10871d3a6e66 k8s.gcr.io/pause:3.6 "/pause" 10 minutes ago Up 10 minutes k8s_POD_kube-apiserver-master_kube-system_01ecc291decf2feb7dae990cc7eb8cb6_7 ``` In **COMMAND** column it says “/pause”. I tried to unpause it but no luck ☹ ``` root@master:~# docker unpause 10871d3a6e66 Error response from daemon: Container 10871d3a6e66c897af6cfa33ea4c45668045ab15bb29beab938556937589e3ad is not paused ``` ---
**Author:** ![fox-md](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/fox-md/32/10613_2.png) [@fox-md](https://discuss.kubernetes.io/u/fox-md)\ **Post date:** [January 18, 2024, 8:28pm UTC](https://discuss.kubernetes.io/t/kube-apiserver-is-gone-the-connection-to-the-server-master-6443-was-refused-did-you-specify-the-right-host-or-port/26754/3 "2024-01-18T20:28:11Z")
Hi, Try checking /var/log/pods directory. You might find kubeapi server logs. Otherwise check kubelet logs. ---
**Author:** ![czezz](https://avatars.discourse-cdn.com/v4/letter/c/54ee81/32.png) [@czezz](https://discuss.kubernetes.io/u/czezz)\ **Post date:** [January 23, 2024, 12:46pm UTC](https://discuss.kubernetes.io/t/kube-apiserver-is-gone-the-connection-to-the-server-master-6443-was-refused-did-you-specify-the-right-host-or-port/26754/4 "2024-01-23T12:46:15Z")
**[WORKAROUND]** I could not find any solution for my problem. As this is just my VirtualBox LAB system (and **I do not mind the data/projects to lose** ), I have re-initiated it (cluster). 1. Reset master node: ```auto sudo su - kubeadm reset -f rm -rf /etc/cni /etc/kubernetes /var/lib/dockershim /var/lib/etcd /var/lib/kubelet /var/run/kubernetes ~/.kube/* iptables -F && iptables -X iptables -t nat -F && iptables -t nat -X iptables -t raw -F && iptables -t raw -X iptables -t mangle -F && iptables -t mangle -X systemctl restart docker exit rm -rf ~/.kube/* ``` 1. Run kubeadm init ```auto sudo kubeadm init --ignore-preflight-errors=NumCPU --control-plane-endpoint master:6443 --pod-network-cidr 10.10.0.0/16 ``` 1. Add certificates ($HOME/.kube/config) ```auto mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config ``` 1. Apply CNI (in my case calico) ```auto kubectl apply -f calico.yaml kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready control-plane,master 29m v1.23.4 ``` Rejoin Node/Worker: 1. Make sure to have master entry in /etc/hosts 2. Re-join the node: ```auto sudo kubeadm reset sudo kubeadm join master:6443 --token aaaaaa.aaaaaaaaaaaaaaaa \ --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ``` ---
**Author:** ![mohitkamboj2799](https://avatars.discourse-cdn.com/v4/letter/m/ee7513/32.png) [@mohitkamboj2799](https://discuss.kubernetes.io/u/mohitkamboj2799)\ **Post date:** [February 1, 2024, 11:48am UTC](https://discuss.kubernetes.io/t/kube-apiserver-is-gone-the-connection-to-the-server-master-6443-was-refused-did-you-specify-the-right-host-or-port/26754/5 "2024-02-01T11:48:00Z")
check container exist log using docker logs container id then see what is the error