How to disable token based authentication and use certificates instead (hardening k8s)

Hello everybody,

We getting close to going to production with our new clusters based on Microk8s.
Currently we’re hardening the clusters according to the CIS benchmark: https://github.com/aquasecurity/kube-bench

Right now we’re looking at disabling token based authentication, which is enabled by default in Microk8s.

When I remove the kube-apiserver argument:
--token-auth-file=${SNAP_DATA}/credentials/known_tokens.csv
and optionally delete the known_tokens.csv file then token based access no longer works.
So far so good.

I could also create new users that use certificates according to this documentation:
https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#normal-user
although I expect to use AD-integration via oauth2-proxy instead for normal users.

So, my questions are these:

  • When disabling tokens, what about the existing users defined in known_tokens.csv?
  • Do I try to update the admin user so it authenticates via certificate instead?
    • If yes, how?
    • If no, should the clusterrole be deleted? I guess it can’t be used anymore if so.
    • Should I use the more commonly used cluster-admin for a new admin user?
  • and what about the remaining entries in the file. Will proxy, controller-manager, kubelet and scheduler still work or should their authentication be updated to use certificates?
    • Kubelet at least must use certificates via the setting the kubelet arguments: –tls-cert-file and –tls-private-key-file

Btw: I’ve created this issue to have the default installation be more CIS compliant by default:
https://github.com/ubuntu/microk8s/issues/2517

Thanks in advance for any insight!
Best regards
Jesper Berggren

Well, in the lack of responses, here’s our solution (if anyone else is interested):

Setup TLS bootstrapping for kubelet as described in: TLS bootstrapping | Kubernetes
This creates a bootstrap that’ll generate client certificates for kubelet.
Remember to set the following arguments in kubelet config:

--rotate-certificates=true
--rotate-server-certificates=true
--tls-cert-file=${SNAP_DATA}/certs/kubelet-server-current.pem
--tls-private-key-file=${SNAP_DATA}/certs/kubelet-server-current.pem

and set this in kube-apiserver:
--kubelet-certificate-authority=${SNAP_DATA}/certs/ca.crt

Since bootstrapping of certificates is not (yet?) possible for scheduler, kube-controller and proxy then manually create certificates for them including the admin user:

cd /var/snap/microk8s/current/certs

openssl genrsa -out admin.key 2048
openssl req -new -key admin.key -subj "/CN=admin/O=system:masters" -out admin.csr
openssl x509 -req -in admin.csr -CA ca.crt -CAkey ca.key -CAcreateserial  -out admin.crt -days 1000

openssl genrsa -out kube-controller-manager.key 2048
openssl req -new -key kube-controller-manager.key -subj "/CN=system:kube-controller-manager" -out kube-controller-manager.csr
openssl x509 -req -in kube-controller-manager.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out kube-controller-manager.crt -days 1000

openssl genrsa -out kube-proxy.key 2048
openssl req -new -key kube-proxy.key -subj "/CN=system:kube-proxy" -out kube-proxy.csr
openssl x509 -req -in kube-proxy.csr -CA ca.crt -CAkey ca.key -CAcreateserial  -out kube-proxy.crt -days 1000

openssl genrsa -out kube-scheduler.key 2048
openssl req -new -key kube-scheduler.key -subj "/CN=system:kube-scheduler" -out kube-scheduler.csr
openssl x509 -req -in kube-scheduler.csr -CA ca.crt -CAkey ca.key -CAcreateserial  -out kube-scheduler.crt -days 1000

Replace tokens in config files (client.config, controller.config, proxy.config and scheduler.config):
cd /var/snap/microk8s/current/credentials
Replace last line containing token with (example for admin, which also could be done in ~/.kube/config):

users:
- name: admin
  user:
    client-certificate: /var/snap/microk8s/current/certs/admin.crt
    client-key: /var/snap/microk8s/current/certs/admin.key

Disable token based access:
vi /var/snap/microk8s/current/args/kube-apiserver
Delete or comment out this line:
--token-auth-file=${SNAP_DATA}/credentials/known_tokens.csv

Restart cluster and you’re done.

Hope that helps somebody :slight_smile:

Best regards
Jesper Berggren