Kubectl not able to use refresh-token

General Discussions microk8s
, ,
1

Hi,
I am having a OIDC setup for authorization to kubernetes GKE cluster. I have kube-oidc-proxy in front of my GKE cluster, which I use as reverse proxy to have group based authentication.
For OIDC provider, I am using dex with GOOGLE as IDP and dex-authenticator (inplace of gangway.)
I’m using a wildcard certificate issued for “*.xyz.in” . using this wildcard, I have created a DNS record named dex.xyz.in for https request.

Everything works fine initially. i am able to access dex service (dex.xyz.in) using dex-authenticator, it generates suitable kubeconfig file with server flag pointing to kube-oidc-proxy service of LB type, in certificate-authority-data, I use ca.crt value of *.xyz.in as i will be using this same TLS everywhere. i have used this TLS in dex configmap as well and have applied trusted_ca_root config in dex-authenticator with value same as that in kubeconfig.

When i run kubectl commands untill id-token gets expiry, everything works fine. But as soon as id-token get expired, kubect get pods (one example) gives:
Unable to connect to the server: Get “https://dex.xyz.in:32000/.well-known/openid-configuration”: x509: certificate signed by unknown authority

This is happening only when refresh-token must be used to get new id-token. Not able to understand why this is happening as i am able to access this URL over chrome and also curl request
curl --request POST --data ‘client_id=dex-auth&client_secret=<abc>&refresh_token=<pqr>&grant_type=refresh_token’ https://dex.xyz.in:32000/token works propely and returns appropriate results but kubectl is not able to work with this.

I encountered similar error with dex-authenticator when I was not giving trusted_root_ca field in its configurations. But once I entered that filed, dex-authenticator worked fine.

My kubectl version:

Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.10-gke.2000", GitCommit:"0823380786b063c3f71d5e7c76826a972e30550d", GitTreeState:"clean", BuildDate:"2022-03-17T09:22:22Z", GoVersion:"go1.16.14b7", Compiler:"gc", Platform:"linux/amd64"}

My dex conigmap:

issuer: https://dex.xyz.in:32000 
storage:
  type: kubernetes
  config:
    inCluster: true
web:
  https: 0.0.0.0:5556
  tlsCert: /etc/dex/tls.crt  (corresponds to tls.crt issued for the wildcard certificate issued to .xyz.in)
  tlsKey: /etc/dex/tls.key (corresponds to tls.key issued for the wildcard certificate issued to .xyz.in)
connectors:
- type: google
  id: google
  name: Google
  config:
    clientID: google_client_id
    clientSecret: google_client_secret
    redirectURI: https://dex.xyz.in:32000/callback
    org: kubernetes
    serviceAccountFilePath: /etc/dex/dex-googleauth/googleauth.json
    adminEmail: adminEmail
expiry:
  idTokens: 2m
  refreshTokens:
    absoluteLifetime: 876000h
    reuseInterval: 3s
    validIfNotUsedFor: 4392h
oauth2:
  skipApprovalScreen: false
  responseTypes: ["code", "token", "id_token"]

staticClients:
- id: dex-auth
  redirectURIs:
  - 'http://10.x.x.x:5555/callback'
  name: 'Dex-Authenticator'
  secret: dex-client-secret

enablePasswordDB: true
staticPasswords:
- email: "admin@example.com"
  # bcrypt hash of the string "password": $(echo password | htpasswd -BinC 10 admin | cut -d: -f2)
  hash: "$2a$10$2b2cU8CPhOTaGrs1HRQuAueS7JTT5ZHsHSzYiFPm1leZck7Mc8T4W"
  username: "admin"
  userID: "08a8684b-db88-4b73-90a9-3cd1661f5466"

dex-authenticator configmap:

listen: http://0.0.0.0:5555
web_path_prefix: /
trusted_root_ca: ca.crt for the wildcard certificate issued to .xyz.in
clusters:
- client_id: dex-auth
  client_secret: dex-client-secret
  description: testing cluster for oidc
  issuer: https://dex.xyz.in:32000
  name: test
  redirect_uri: http://10.x.x.x:5555/callback
  short_description: testing cluster for oidc

In kube-oidc-proxy, i am using secrets file to give values such as

  1. oidc.client-id: same as client_id in dex-authenticator clusters config.
  2. oidc.groups-claim
  3. oidc.issue-url: https://dex.xyz.in:32000
  4. oidc.ca-pem: using ca.crt issued for .xyz.in, similar to how i am using it in trusted_root_ca in dex-authenticator configmap.

Kindly point out error, been trying to debug this for a long time now.