Define multiple kube-schedulers with Kubernetes version 1.22

General Discussions
1

You can format your yaml by highlighting it and pressing Ctrl-Shift-C, it will make your output easier to read.

How can we simply achieve $subject?

I defined the following based on the official documentation which is yet to be updated.

Scheduler workload and RBAC

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-scheduler
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: my-scheduler-as-kube-scheduler
subjects:
- kind: ServiceAccount
  name: my-scheduler
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: system:kube-scheduler
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: my-scheduler-as-volume-scheduler
subjects:
- kind: ServiceAccount
  name: my-scheduler
  namespace: kube-system
roleRef:
  kind: ClusterRole
  name: system:volume-scheduler
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    component: scheduler
    tier: control-plane
  name: my-scheduler
  namespace: kube-system
spec:
  selector:
    matchLabels:
      component: scheduler
      tier: control-plane
  replicas: 1
  template:
    metadata:
      labels:
        component: scheduler
        tier: control-plane
        version: second
    spec:
      serviceAccountName: my-scheduler
      containers:
      - command:
        - /usr/local/bin/kube-scheduler
        - --bind-address=0.0.0.0
        - --leader-elect=false
        - --config=/etc/kubernetes/my-scheduler/my-scheduler-config.yaml
        - --port=0
        image: k8s.gcr.io/kube-scheduler:v1.22.0
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10259
            scheme: HTTPS
          initialDelaySeconds: 15
        name: kube-second-scheduler
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10259
            scheme: HTTPS
        resources:
          requests:
            cpu: '0.1'
        securityContext:
          privileged: false
        volumeMounts:
        - name: config-volume
          mountPath: /etc/kubernetes/my-scheduler
      hostNetwork: false
      hostPID: false
      volumes:
      - name: config-volume
        configMap:
          # Provide the name of the ConfigMap containing the files you want
          # to add to the container
          name: my-scheduler-config

Also, since the --scheduler-name option has been removed, defined the following Kubernetes Scheduler Configuration simply defining the new name and added it via a ConfigMap, which is pointed to by --config argument.

apiVersion: kubescheduler.config.k8s.io/v1beta2
kind: KubeSchedulerConfiguration
profiles:
  - schedulerName: my-scheduler

But a Pod scheduled remains in Pending state when pointed to use the above created scheduler.

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: my-scheduler-check
  name: my-scheduler-check
spec:
  schedulerName: my-scheduler
  containers:
  - image: nginx
    name: my-scheduler-check
  dnsPolicy: ClusterFirst
  restartPolicy: Always

I can see the following logs in the scheduler which does not highlight any issue (had to provide RBAC based permissions to read and access ConfigMaps for the ClusterRole system:kube-scheduler).

Screenshot 2021-10-09 at 11.26.58
Screenshot 2021-10-09 at 11.26.58790×66 4.79 KB 

I1008 21:21:55.514782       1 serving.go:347] Generated self-signed cert in-memory
W1008 21:21:56.074915       1 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
W1008 21:21:56.111405       1 node_prefer_avoid_pods.go:93] NodePreferAvoidPods plugin is deprecated and will be removed in a future version; use node taints instead
W1008 21:21:56.113531       1 authorization.go:47] Authorization is disabled
W1008 21:21:56.113553       1 authentication.go:47] Authentication is disabled
I1008 21:21:56.113565       1 deprecated_insecure_serving.go:51] Serving healthz insecurely on [::]:10251
I1008 21:21:56.122252       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
I1008 21:21:56.122281       1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController
I1008 21:21:56.122285       1 configmap_cafile_content.go:201] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I1008 21:21:56.122308       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I1008 21:21:56.122336       1 configmap_cafile_content.go:201] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
I1008 21:21:56.122354       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I1008 21:21:56.122737       1 secure_serving.go:195] Serving securely on [::]:10259
I1008 21:21:56.122850       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I1008 21:21:56.223270       1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController
I1008 21:21:56.223276       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I1008 21:21:56.223277       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I1008 21:21:56.223364       1 leaderelection.go:248] attempting to acquire leader lease kube-system/kube-scheduler...