Unable to Delete Namespace on Kubernetes 1.28.2 - status still Terminating

General Discussions
5

I was able to resolve the error.

Here are the steps I took:

  1. I listed the resources that have a namespaced scope (i.e., those that are not global to the cluster):

    LIST=$(kubectl api-resources --verbs=list --namespaced -o name | tr "\n" " ")

  2. I checked for any CRDs that are in error among this list:

    for elt in $(echo $LIST); do kubectl get --show-kind --ignore-not-found -n <ANY terminating ns> $elt; done

  3. I found that inferenceservices.serving.kserve.io is causing an issue, so I ran a describe command:

    kubectl describe crd inferenceservices.serving.kserve.io

Message: could not list instances: unable to find a custom resource client for [inferenceservices.serving.kserve.io]: unable to load root certificates: unable to parse bytes as PEM block
Reason: InstanceDeletionFailed
Status: True
Type: Terminating

There is a error message, and a Terminating step.
This is the same issue here : fail installing KServe · Issue #2349 · kubeflow/manifests · GitHub

  1. Since this resource is in a terminating state and is used in my namespace, I removed its finalizer and then deleted the resource:
    kubectl patch crd/inferenceservices.serving.kserve.io -p '{"metadata":{"finalizers":[]}}' --type=merge
kubectl delete crd inferenceservices.serving.kserve.io

After completing these steps, I successfully removed all terminating namespaces.