I had the same issue every time i reboot the system in my Debian master node so i realized that ‘swapoff -a’ was a volatile command. Then i saw when the system comes up that it truly has a swap partition on, despite i comment this out on /etc/fstab and i disabled it and deleted it. My surprise came when i issue the ‘swapon -s’ and it showed me /dev/sda12 as a swap enabled.
‘systemctl status /dev/sda12’ showed me it up and running. (also i used ‘lsblk -f’)
This partition was outside of my Debian partition so i launched GParted software from my Debian partition (gnome) and delete it. No more problems.
Hi ,
You can try “systemctl enable docker” inorder to turn on the docker. Before that, you are supposed to reboot/restart the terminal and try the above command.
Thank you! Have a great day! Let me know if it works.
Hi tania,
Thank you for the response. Unfortunately, it’s still the same. Please, find the attached screenshot.
Thanks
BK
Thanks it works for me
Hi there, if you are still having this issue as I did, there are quite a few things that could be wrong. Ensure swap is off definitely as advised by other comments. Other interesting causes from my sujjourn include.
- The Kubeconfig environmental variable is probably not set.
export KUBECONFIG=/etc/kubernetes/admin.conf or $HOME/.kube/config - The user’s $HOME directory has no .kube/config file.
If you don’t have a .kube or config file
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf HOME/.kube/config sudo chown (id -u):$(id -g) $HOME/.kube/config
Alternatively you can also export KUBECONFIG variable like this:
export KUBECONFIG=$HOME/.kube/config - The server/port configured in the config file above is wrong.
Is it the same as the IP/hostname of the master server? if not did you copy it? You might
want to fix that.
By the way you can get the hostname by issueing the hostname command on your cli. or
ifconfig for the ip. - Kubelet service may be down. This may be due to the fact that swap is enabled.
sudo swapoff -a
To make it permanent go to /etc/fstab
sudo -i
swapoff -a
exit
strace -eopenat kubectl version
sudo systemctl restart kubelet.service - Docker service may be down, hence the kubeapi pod isn’t running
sudo systemctl start docker
sudo systemctl start kubelet
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown (id -u):(id -g) $HOME/.kube/config
- Firewalls may be blocking the access
sudo systemctl status firewalld #redhat centos
sudo systemctl stop firewalld #redhat, centos
sudo ufw status verbose #ubuntu
sudo ufw disable #ubuntu
check my response below
Thanks! That saved me four and a half days of troubleshooting!
Disk space was the cause in my case as well.
When I did “df -h”, no overlay or shm (mounted on /var/lib/docker…) was listed, until I increased the free disk space.
I came here for a similar problem and while your post didnt directly solve my issue, it helped guide me to where my issue was.
In my case on reboot my kubelet service was down, and nothing was starting due to swap being-on even though I disabled it.
You mention how to disable it but it doesn’t work.
I think a lot of people are following the same guides, and the issue is that the guides are all the exact same and contain the same snippet of cause which is causing this problem. These 3 lines need to be run individually, and not strung together:
sudo dphys-swapfile swapoff && \
sudo dphys-swapfile uninstall && \
sudo systemctl disable dphys-swapfile
remove the && and \ and copy and paste each line in the command line, this will ensure swap is disabled and persistent across a reboot.
Please try
strace -eopenat kubectl version
either using sudo or root.
the above command worked for me. i too banged head a lot
i have deployed Kubernetes on Ubuntu16.04 on Virtualbox
Hi , my k8s occurs this problem because CentOS used the SWAP.
And this fixes my problem:
in the shell:
swap -a
and then:
edit the file: /etc/fstab, you will see a line like below:
/dev/mapper/centos-swap swap swap defaults 0 0
just disable it :
# /dev/mapper/centos-swap swap swap defaults 0 0
and back to the shell:
free -m
you will see the SWAP -0 -0.
and then reboot the OS. I hope it helps.
Thanks so much! This worked beautifully! Tremendously appreciated.
This worked for me.
sudo cp /etc/kubernetes/admin.conf $HOME/
sudo chown (id -u):(id -g) $HOME/admin.conf
export KUBECONFIG=$HOME/admin.conf
Just checked kubelet.service is running or not.
I have faced same issue and resolved by just start of kubelet.service . PFB.
[root@ip-297-49-42-100 ~]# kubectl get node
The connection to the server 297.49.42.100:6443 was refused - did you specify the right host or port?
[root@ip-297-49-42-100 ~]# systemctl status kubelet.service
Display all 494 possibilities? (y or n)
[root@ip-297-49-42-100 ~]# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; disabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: inactive (dead)
Docs: https://kubernetes.io/docs/
[root@ip-297-49-42-100 ~]# systemctl start kubelet.service
[root@ip-297-49-42-100 ~]# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; disabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Mon 2020-04-20 14:24:34 UTC; 2s ago
Docs: https://kubernetes.io/docs/
Main PID: 5115 (kubelet)
Tasks: 14
Memory: 94.1M
CGroup: /system.slice/kubelet.service
└─5115 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet…
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: W0420 14:24:35.882212 5115 pod_container_deletor.go…ners
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: I0420 14:24:35.882289 5115 kubelet_node_status.go:2…tach
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: I0420 14:24:35.900996 5115 kubelet_node_status.go:2…tach
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: W0420 14:24:35.919094 5115 pod_container_deletor.go…ners
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:35.936701 5115 kubelet.go:2267] node "i…ound
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: W0420 14:24:36.014904 5115 status_manager.go:556] Faile…
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:36.036878 5115 kubelet.go:2267] node "i…ound
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:36.136972 5115 kubelet.go:2267] node "i…ound
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:36.214917 5115 kubelet_node_status.go:9…used
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:36.237347 5115 kubelet.go:2267] node "i…ound
Hint: Some lines were ellipsized, use -l to show in full.
[root@ip-297-49-42-100 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
ip-297-49-42-100.us-east-2.compute.internal NotReady master 65m v1.18.2
[root@ip-297-49-42-100 ~]# systemctl enable kubelet.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@ip-297-49-42-100 ~]#
Just checked the status of kubelet.service.
Mostly this issue occurs when this service does not enable on boot. PFB.
[root@ip-297-49-42-100 ~]# kubectl get node
The connection to the server 297.49.42.100:6443 was refused - did you specify the right host or port?
[root@ip-297-49-42-100 ~]# systemctl status kubelet.service
Display all 494 possibilities? (y or n)
[root@ip-297-49-42-100 ~]# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; disabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: inactive (dead)
Docs: https://kubernetes.io/docs/
[root@ip-297-49-42-100 ~]# systemctl start kubelet.service
[root@ip-297-49-42-100 ~]# systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; disabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Mon 2020-04-20 14:24:34 UTC; 2s ago
Docs: https://kubernetes.io/docs/
Main PID: 5115 (kubelet)
Tasks: 14
Memory: 94.1M
CGroup: /system.slice/kubelet.service
└─5115 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet…
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: W0420 14:24:35.882212 5115 pod_container_deletor.go…ners
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: I0420 14:24:35.882289 5115 kubelet_node_status.go:2…tach
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: I0420 14:24:35.900996 5115 kubelet_node_status.go:2…tach
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: W0420 14:24:35.919094 5115 pod_container_deletor.go…ners
Apr 20 14:24:35 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:35.936701 5115 kubelet.go:2267] node "i…ound
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: W0420 14:24:36.014904 5115 status_manager.go:556] Faile…
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:36.036878 5115 kubelet.go:2267] node "i…ound
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:36.136972 5115 kubelet.go:2267] node "i…ound
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:36.214917 5115 kubelet_node_status.go:9…used
Apr 20 14:24:36 ip-297-49-42-100.us-east-2.compute.internal kubelet[5115]: E0420 14:24:36.237347 5115 kubelet.go:2267] node "i…ound
Hint: Some lines were ellipsized, use -l to show in full.
[root@ip-297-49-42-100 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
ip-297-49-42-100.us-east-2.compute.internal NotReady master 65m v1.18.2
[root@ip-297-49-42-100 ~]# systemctl enable kubelet.service
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@ip-297-49-42-100 ~]#
-
systemctl status kubelet -> it should be runnning
-
kubeadm reset -> reset the kubeadm using this command
Your Kubernetes control-plane has initialized successfully! -
Now try to RUN “kubectl get pods”
kubectl cluster-info
To further debug and diagnose cluster problems, use ‘kubectl cluster-info dump’.
The connection to the server 100.0.0.1:6443 was refused - did you specify the right host or port?
vagrant@master:~$ kubectl get nodes
The connection to the server 100.0.0.1:6443 was refused - did you specify the right host or port?
i am still facing the same issue. any solution pls ?
Ensure that your docker service is running on all of your nodes and try “kubeclt cluster-info dump” this resolved my issue!
should read sudo chown $(id -u):$(id -g) $HOME/admin.conf
Need dollar signs before those two open parens.
I am facing same issue.any sugesstions.
kubectl get nodes
The connection to the server :6443 was refused - did you specify the right host or port?