Cluster information:
Kubernetes version: 27.3
Cloud being used: Azure (AKS)
Installation method:Managed
Host OS: Linux
CNI and version:
CRI and version:
Our AKS (Linux + CPU) has been working with Azure KeyVault using managed identities for a while, and secrets are being mounted to the CSI volumes with no problems. Recently we have added GPU nodes to the cluster, and only the PODs and containers running on GPU nodes don’t have CSI volumes mounted and hence no access to KeyVault secrets. Containers on GPU nodes fail to start as a result.
Is there anything we need to do to make the CSI volumes mounted onto the PODs running on GPU nodes? (I have tried reinstalling the secret provides with no luck)
Here is a health POD running on CPU:
Volumes: │
│ analytics-api-azure-keyvault-volume: │
│ Type: CSI (a Container Storage Interface (CSI) volume source) │
│ Driver: secrets-store.csi.k8s.io │
│ FSType: │
│ ReadOnly: true │
│ VolumeAttributes: secretProviderClass=azure-es-analytics-api-vault │
│ kube-api-access-zsjk8: │
│ Type: Projected (a volume that contains injected data from multiple sources) │
│ TokenExpirationSeconds: 3607 │
│ ConfigMapName: kube-root-ca.crt │
│ ConfigMapOptional: │
│ DownwardAPI: true
and here is a POD running on GPU with no CSI volume.
Volumes: │
│ kube-api-access-w95j9: │
│ Type: Projected (a volume that contains injected data from multiple sources) │
│ TokenExpirationSeconds: 3607 │
│ ConfigMapName: kube-root-ca.crt │
│ ConfigMapOptional: │
│ DownwardAPI: true