Troubleshooting KubeSchedulerConfiguration Registration in a Custom Kubernetes Scheduler Plugin

General Discussions
1

Hi,

I’m currently developing a Kubernetes scheduler plugin which, for the time being, performs no action to streamline our conversation. I’ve appended an “influx” parameter to the KubeSchedulerConfiguration, which is structured as follows:

apiVersion: kubescheduler.config.k8s.io/v1
kind: KubeSchedulerConfiguration
clientConnection:
  kubeconfig: /etc/kubernetes/scheduler.conf
profiles:
  - schedulerName: default-scheduler
    pluginConfig:
    - name: SchedulerPlugin
      args:
        influx:
          hostname: customPlugin
          port: 8080

When I replace the Kubernetes’ default scheduler by my scheduler, I encounter the following error: error loading config file "/etc/kubernetes/scheduler-config.yaml": no kind "KubeSchedulerConfiguration" is registered for version "kubescheduler.config.k8s.io/v1" in scheme "pkg/runtime/scheme.go:100"

Nevertheless, I have registered the plugin configuration schemes that correspond with the stated apiVersion (kubescheduler.config.k8s.io/v1).

main.go, which initiates the scheduler:

package main
// skip imports
func main() {
	// ensure that our types are known to K8S (in particular the plugin's config type)
	schedulerPlugin.RegisterSchemes()

	// run the scheduler
	command := app.NewSchedulerCommand(
		app.WithPlugin(schedulerPlugin.Name, schedulerPlugin.New),
	)
	if err := command.Execute(); err != nil {
		fmt.Fprintf(os.Stderr, "%v\n", err)
		os.Exit(1)
	}
}

config.go - where the scheme registration is specified:

package schedulerPlugin
// skip imports
type SchedulerArgs struct {
	metav1.TypeMeta `json:",inline"` // to implement runtime.Object

	Influx InfluxParams `json:"influx"`
}

type InfluxParams struct {
	Hostname string `json:"hostname"`
	Port     string `json:"port"`
}

// to implement runtime.Object
func (in *SchedulerArgs) DeepCopyObject() runtime.Object {
	if in == nil {
		return nil
	}
	copy := *in
	return &copy // perfectly fine in go
}

// --- K8S machinery to register the config's type ---

// Register kube-scheduler schemes.
// This must be called in main, before trying to create the scheduler command.
func RegisterSchemes() {
	buildScheme(&schedconfig.SchemeBuilder, runtime.APIVersionInternal)
	buildScheme(&schedconfigv1.SchemeBuilder, "v1")
	println("Register scheme for v1, internal.")
}

func buildScheme(builder *runtime.SchemeBuilder, version string) {
	// register a function that will add knowledge about our types to K8S
	builder.Register(func(s *runtime.Scheme) error {
		addKnownTypes(s, version)
		return nil
	})
	// applies the function to the kube-scheduler scheme, which handles the scheduler config
	util.Must(builder.AddToScheme(kubeschedulerscheme.Scheme))
}

func addKnownTypes(scheme *runtime.Scheme, version string) {
	schemeGroupVersion := schema.GroupVersion{
		Group:   schedconfig.GroupName,
		Version: version,
	}
	scheme.AddKnownTypes(
		// first, the version
		schemeGroupVersion,
		// then one or more pointers to the structs we want to register
		&SchedulerArgs{},
	)
}

Am I missing a step in my registration process that could be causing this error?

Cluster information:

Kubernetes version: 1.29.3
Cloud being used: Baremetal
Installation method: Kubeadm @ 1.29.3
Host OS: Debian 12
CNI and version: calico 3.25
CRI and version: containerd 1.6.28

2 Likes
2

Hello once more,

I’ve managed to resolve the issue: a complete reinstallation of Kubernetes along with the kubeadm, kubelet, and kubectl packages did the trick.

I don’t really know what was the root cause of the problem with my initial setup