The connection to the server <host>:6443 was refused - did you specify the right host or port?

kubeadm reset
Use above mentioned command on master & node servers solved all my problems.

Master Server: Kali Linux (VM)
swapoff -a
rm /etc/kubernetes/kubelet.conf
rm /etc/kubernetes/pki/ca.crt
systemctl daemon-reload
systemctl start containerd
systemctl start docker
systemctl enable docker
systemctl start kubelet
systemctl enable kubelet
kubeadm reset
kubeadm init
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Node Server 1: CentOS (VM)
swapoff -a
rm /etc/kubernetes/kubelet.conf
rm /etc/kubernetes/pki/ca.crt
systemctl daemon-reload
systemctl start containerd
systemctl enable containerd
systemctl start docker
systemctl enable docker
systemctl start kubelet
systemctl enable kubelet
kubeadm reset
kubeadm join ENTER_MASTER_SERVER_PUBLIC_IP:6443 --token xjcvh0.zyn31te4go5sj55n --discovery-token-ca-cert-hash sha256:064d364a596f04aca2d15c3544cf9774a66041d46f50a28c78bda4756e9f78c2

Node Server 2: Alma Linux (VM)
swapoff -a
rm /etc/kubernetes/kubelet.conf
rm /etc/kubernetes/pki/ca.crt
systemctl daemon-reload
systemctl start containerd
systemctl start docker
systemctl enable docker
systemctl start kubelet
systemctl enable kubelet
kubeadm reset
kubeadm join ENTER_MASTER_SERVER_PUBLIC_IP:6443 --token xjcvh0.zyn31te4go5sj55n --discovery-token-ca-cert-hash sha256:064d364a596f04aca2d15c3544cf9774a66041d46f50a28c78bda4756e9f78c2

Remove IP Restrictions:
iptables -F && sudo iptables -t nat -F && sudo iptables -t mangle -F && sudo iptables -X
Use above mentioned command on all servers to allow all ips & ports if any error occurs related to connection refused. This is very insecure to allow all ips and ports, use above only for testing purpose.

Use above mentioned link to allow specific ports on master and node servers if any error occurs related to connection refused.