Kubernetes version: 1.18.3
Cloud being used: bare-metal
Installation method: docker
Host OS: RHEL 7.8
One for the brains trust.
I am trying to init a k8s instance with keycloak oidc configurations.
At the moment this is a play instance that is hosted on docker and consists of 1 server & 2 nodes.
The node etcd1 is the master on, the host oss is hosting keycloak. I can issue a curl command from this host to get a token from the keycloak server.
For the purposes of this deployment there is not firewalld or swap configured, the server is not running Selinux in enforcing mode.
I have created the following config file
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress:
bindPort: 6443
nodeRegistration:
criSocket: /var/run/dockershim.sock
name: etcd1.
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
—
apiServer:
timeoutForControlPlane: 4m0s
extraArgs:
authorization-mode: RBAC
oidc-client-id: k8s
oidc-issuer-url: https://oss.:8443/auth/realms/
oidc-username-claim: preferred_username
oidc-username-prefix: ‘-’
oidc-groups-claim: groups
oidc-groups-prefix: ‘k8s:’
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.18.3
networking:
dnsDomain: cluster.local
serviceSubnet: 10.96.0.0/12
scheduler: {}
When I run this config using “kubeadm init --config file.yaml” without the oidc authorization-mode extrArgs options. I end up with a running kubenetes instance.
When I run “kube init --config file.yaml --v=6” with the extraArgs oicd/authorisation-mode data still in the config, i get the following errors
I0528 04:55:10.583732 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/healthz?timeout=10s in 0 milliseconds
I0528 04:55:11.085481 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/healthz?timeout=10s in 2 milliseconds
I0528 04:55:15.866910 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/healthz?timeout=10s 500 Internal Server Error in 4282 milliseconds
I0528 04:55:16.085179 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds
I0528 04:55:16.584967 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/healthz?timeout=10s 500 Internal Server Error in 1 milliseconds
I0528 04:55:17.086526 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/healthz?timeout=10s 500 Internal Server Error in 3 milliseconds
I0528 04:55:17.585106 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/healthz?timeout=10s 200 OK in 1 milliseconds
[apiclient] All control plane components are healthy after 14.003540 seconds
I0528 04:55:17.585199 22004 uploadconfig.go:108] [upload-config] Uploading the kubeadm ClusterConfiguration to a ConfigMap
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0528 04:55:17.589500 22004 round_trippers.go:443] POST https://<IP ADDRESS>:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 2 milliseconds
I0528 04:55:17.593013 22004 round_trippers.go:443] POST https://<IP ADDRESS>:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 2 milliseconds
I0528 04:55:17.596637 22004 round_trippers.go:443] POST https://<IP ADDRESS>:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 2 milliseconds
I0528 04:55:17.597257 22004 uploadconfig.go:122] [upload-config] Uploading the kubelet component config to a ConfigMap
[kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster
I0528 04:55:17.603961 22004 round_trippers.go:443] POST https://<IP ADDRESS>:6443/api/v1/namespaces/kube-system/configmaps?timeout=10s 201 Created in 5 milliseconds
I0528 04:55:17.606654 22004 round_trippers.go:443] POST https://<IP ADDRESS>:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/roles?timeout=10s 201 Created in 2 milliseconds
I0528 04:55:17.609042 22004 round_trippers.go:443] POST https://<IP ADDRESS>:6443/apis/rbac.authorization.k8s.io/v1/namespaces/kube-system/rolebindings?timeout=10s 201 Created in 2 milliseconds
I0528 04:55:17.609164 22004 uploadconfig.go:127] [upload-config] Preserving the CRISocket information for the control-plane node
I0528 04:55:17.609175 22004 patchnode.go:30] [patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "etcd1.<DOMAIN>" as an annotation
I0528 04:55:18.111357 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/api/v1/nodes/etcd1.<DOMAIN>?timeout=10s 404 Not Found in 1 milliseconds
I0528 04:55:18.611226 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/api/v1/nodes/etcd1.<DOMAIN>?timeout=10s 404 Not Found in 1 milliseconds
I0528 04:55:19.111370 22004 round_trippers.go:443] GET https://<IP ADDRESS>:6443/api/v1/nodes/etcd1.<DOMAIN>?timeout=10s 404 Not Found in 1 milliseconds
when I run curl against the URL https://:6443/api/v1/nodes/etcd1.?timeout=10s
curl -k -Ssl https://<IP ADDRESS>:6443/api/v1/nodes/etcd1.<DOMAIN>?timeout=10s
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "nodes \"etcd1.<DOMAIN>\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope",
"reason": "Forbidden",
"details": {
"name": "etcd1.<DOMAIN>",
"kind": "nodes"
},
"code": 403
journalctl -xeu kubelet
May 28 05:07:08 etcd1.<DOMAIN> kubelet[22160]: E0528 05:07:08.776615 22160 kubelet.go:2267] node "etcd1.<DOMAIN>" not found
May 28 05:07:08 etcd1.<DOMAIN> kubelet[22160]: E0528 05:07:08.876830 22160 kubelet.go:2267] node "etcd1.<DOMAIN>" not found
May 28 05:07:08 etcd1.<DOMAIN> kubelet[22160]: E0528 05:07:08.977067 22160 kubelet.go:2267] node "etcd1.<DOMAIN>" not found
May 28 05:07:09 etcd1.<DOMAIN> kubelet[22160]: E0528 05:07:09.077190 22160 kubelet.go:2267] node "etcd1.<DOMAIN>" not found
From what I have been able to deduce, is that the host etcd1 is not configured to be able to connect to the kube environment even though it is hosting it.
I am just not sure of how to resolve this.