What happened?
I init a kubernetes cluster in amazon-ec2 by kubeadm.Flannel is it’s network plugin.
kubeadm init --pod-network-cidr=10.244.0.0/16 --kubernetes-version=v1.24.17 --service-cidr=10.96.0.0/16
use default flannel yaml
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
The security rules of ec2. I opened all ports.
create dep and svc
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: test
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
nodeSelector:
kubernetes.io/hostname: k8s-node-wu
containers:
- name: nginx-pod
image: nginx:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
apiVersion: v1
kind: Service
metadata:
name: nginx-svic
namespace: test
spec:
selector:
app: nginx
ports:
- protocol: TCP
port: 80
nodePort: 32080
type: NodePort
kubectl get no -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8s-master Ready control-plane 73d v1.24.17 172.31.22.234 <none> Amazon Linux 2023 6.1.55-75.123.amzn2023.x86_64 containerd://1.7.2
k8s-node2 Ready <none> 73d v1.24.17 172.31.16.122 <none> Amazon Linux 2023 6.1.55-75.123.amzn2023.x86_64 containerd://1.7.2
k8s-node1 Ready <none> 73d v1.24.17 172.31.22.26 <none> Amazon Linux 2023 6.1.55-75.123.amzn2023.x86_64 containerd://1.7.2
k get po -ntest -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-79db656995-shx4q 1/1 Running 0 17h 10.244.2.28 k8s-node1 <none> <none>
kubectl describe svc nginx-svc -ntest
Name: nginx-svic
Namespace: test
Labels: <none>
Annotations: <none>
Selector: app=nginx
Type: NodePort
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.96.28.164
IPs: 10.96.28.164
Port: <unset> 80/TCP
TargetPort: 80/TCP
NodePort: <unset> 32080/TCP
Endpoints: 10.244.2.28:80
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
Pod of the dep is dispatched to k8s-node2 and run successfully. But when I curl Endpoints or NodePort of master, the connect is failed.
curl localhost:32080
curl: (28) Failed to connect to localhost port 32080 after 132843 ms: Couldn't connect to server
curl 10.244.2.28:80
curl: (28) Failed to connect to 10.244.2.28 port 80 after 129717 ms: Couldn't connect to server
curl by k8s-node2’s real ip, successfully.
curl 52.83.66.62:32080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Kubernetes core pods, flannel pod as well as kubelet are Running normally, and no correlative error log.
k get po -nkube-system -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-7f74c56694-ctgnp 1/1 Running 0 73d 10.244.2.2 k8s-node2 <none> <none>
coredns-7f74c56694-rgjg8 1/1 Running 0 73d 10.244.1.2 k8s-node1 <none> <none>
etcd-k8s-master 1/1 Running 8 73d 172.31.22.234 k8s-master <none> <none>
kube-apiserver-k8s-master 1/1 Running 3 73d 172.31.22.234 k8s-master <none> <none>
kube-controller-manager-k8s-master 1/1 Running 3 73d 172.31.22.234 k8s-master <none> <none>
kube-proxy-87btg 1/1 Running 0 71d 172.31.22.26 k8s-node1 <none> <none>
kube-proxy-hfz82 1/1 Running 0 71d 172.31.22.234 k8s-master <none> <none>
kube-proxy-qv2hm 1/1 Running 0 71d 172.31.16.122 k8s-node2 <none> <none>
kube-scheduler-k8s-master 1/1 Running 9 73d 172.31.22.234 k8s-master <none> <none>
k get po -nkube-flannel -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-flannel-ds-9g7m8 1/1 Running 0 71d 172.31.16.122 k8s-node2 <none> <none>
kube-flannel-ds-m7vh2 1/1 Running 0 71d 172.31.22.26 k8s-node1 <none> <none>
kube-flannel-ds-w4hp8 1/1 Running 0 71d 172.31.22.234 k8s-master <none> <none>
Other anomalies have not been found. kubectl logs、exec, etc are executed successfully.
What did you expect to happen?
apply NodePort successfully.
How can we reproduce it (as minimally and precisely as possible)?
see What Happend
Anything else we need to know?
No response
Kubernetes version
$ kubectl version
# paste output here
Cloud provider
OS version
# On Linux:
$ cat /etc/os-release
NAME="Amazon Linux"
VERSION="2023"
ID="amzn"
ID_LIKE="fedora"
VERSION_ID="2023"
PLATFORM_ID="platform:al2023"
PRETTY_NAME="Amazon Linux 2023"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023"
HOME_URL="https://aws.amazon.com/linux/"
BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023"
SUPPORT_END="2028-03-01"
$ uname -a
Linux k8s-master 6.1.55-75.123.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Sep 26 20:06:16 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Install tools
Container runtime (CRI) and version (if applicable)
ctr -v
ctr github.com/containerd/containerd 1.7.2