# Addon: cert-manager

**URL:** https://discuss.kubernetes.io/t/addon-cert-manager/21122
**Category:** microk8s
**Tags:** docs
**Created:** [August 29, 2022, 11:40am UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122 "2022-08-29T11:40:31Z")
**Posts on this page:** 19
**Page:** 1

<div class="post-metadata">

### Author: ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)
#### Post date: [August 29, 2022, 11:40am UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/1 "2022-08-29T11:40:31Z")

</div>

From 1.25

This addon installs [Cert Manager](https://cert-manager.io). Cert-Manager is the de-facto standard solution for certificate management in Kubernetes clusters. It supports x.509 certificate management for Kubernetes and OpenShift clusters, retrieving certificates from private (internal) or public issuers, and ensures they are properly rotated and kept up to date.

Install this addon with:

```bash
microk8s enable cert-manager

```

## Automatically generating Let’s Encrypt certificates for Ingress

One of the common use-cases of Cert-Manager is to configure [Kubernetes Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) resources with automatic TLS certificates from Let’s Encrypt.

### Requirements

1. A MicroK8s cluster with a public IP address. This is required to complete the HTTP challenges of Let’s Encrypt.

2. A hostname that resolves to your public IP address, e.g. `my-service.example.com`.

3. A properly configured ingress class for your MicroK8s cluster. The simplest way to do this is to use the [ingress addon](https://discuss.kubernetes.io/t/addon-ingress/11259):

```bash
microk8s enable ingress dns

```

### Create a ClusterIssuer

A `ClusterIssuer` resource is used to configure an account with Let’s Encrypt. All you need is an email address (make sure to use a valid email address).

Create a ClusterIssuer called `lets-encrypt` with the command below. Make sure to replace `microk8s@example.com` below with your email. Note that Let’s Encrypt will refuse to register accounts that use the `example.com` domain.

```bash
microk8s kubectl apply -f - <<EOF
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
 name: lets-encrypt
spec:
 acme:
   email: microk8s@example.com
   server: https://acme-v02.api.letsencrypt.org/directory
   privateKeySecretRef:
     # Secret resource that will be used to store the account's private key.
     name: lets-encrypt-priviate-key
   # Add a single challenge solver, HTTP01 using nginx
   solvers:
   - http01:
       ingress:
         ingressClassName: public
EOF

```

Verify that the ClusterIssuer was created successfully with `microk8s kubectl get clusterissuer -o wide`, which should produce output similar to:

```auto
NAME READY STATUS AGE
lets-encrypt True The ACME account was registered with the ACME server 2m19s

```

### Deploy a service

For this example, we will deploy a simple microbot deployment:

```bash
microk8s kubectl create deploy --image cdkbot/microbot:1 --replicas 3 microbot
microk8s kubectl expose deploy microbot --port 80 --type ClusterIP

```

Ensure the service is up and running with `microk8s kubectl get pod,svc`:

```bash
NAME READY STATUS RESTARTS AGE
pod/microbot-b6996696-sbp76 1/1 Running 0 11s
pod/microbot-b6996696-xmplm 1/1 Running 0 11s
pod/microbot-b6996696-8b82c 1/1 Running 0 11s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 16m
service/microbot ClusterIP 10.152.183.134 <none> 80/TCP 3s

```

### Configure ingress

Next, create a Kubernetes ingress resource that forwards requests made to `https://my-service.example.com` to our microbot service.

Note that the `cert-manager.io/cluster-issuer: lets-encrypt` annotation tells Cert-Manager to automatically retrieve TLS certificates for our domain. The following example needs to reference the correct hostnames for your deployment - please substitute appropriately before running this command:

```bash
microk8s kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: microbot-ingress
 annotations:
   cert-manager.io/cluster-issuer: lets-encrypt
spec:
 tls:
 - hosts:
   - my-service.example.com
   secretName: microbot-ingress-tls
 rules:
 - host: my-service.example.com
   http:
     paths:
     - backend:
         service:
           name: microbot
           port:
             number: 80
       path: /
       pathType: Exact
EOF

```

After a while, Cert-Manager will automatically request a certificate from Let’s Encrypt, populate the `microk8s-ingress-tls` with it and configure the ingress. Finally, you should be able to access your service at its fully qualified domain.

---

<div class="post-metadata">

### Author: ![amo.mycena](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/amo.mycena/32/10823_2.png) [@amo.mycena](https://discuss.kubernetes.io/u/amo.mycena)
#### Post date: [September 14, 2022, 9:35am UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/2 "2022-09-14T09:35:33Z")

</div>

Hi, @evilnick  
Is there any way to use under microk8s 1.22 ?  
I look forward to your reply.  
Thanks.

---

<div class="post-metadata">

### Author: ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)
#### Post date: [September 14, 2022, 11:25am UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/3 "2022-09-14T11:25:37Z")

</div>

@amo.mycena thanks for the message. Unfortunately no, this relies on features which were added in 1.25

---

<div class="post-metadata">

### Author: ![amo.mycena](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/amo.mycena/32/10823_2.png) [@amo.mycena](https://discuss.kubernetes.io/u/amo.mycena)
#### Post date: [September 15, 2022, 7:37am UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/4 "2022-09-15T07:37:55Z")

</div>

Thanks for your reply.

---

<div class="post-metadata">

### Author: ![bramo](https://avatars.discourse-cdn.com/v4/letter/b/f05b48/32.png) [@bramo](https://discuss.kubernetes.io/u/bramo)
#### Post date: [October 11, 2022, 11:04am UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/5 "2022-10-11T11:04:30Z")

</div>

Hi this worked great. I presume I don’t need to keep the microbot deployment running?

Will cert manager get a new TLS cert for any ingress configured with that annotation, or do I have to keep this ingress and just reconfigure the backend service for my ‘real’ website?

---

<div class="post-metadata">

### Author: ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)
#### Post date: [October 11, 2022, 2:04pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/6 "2022-10-11T14:04:02Z")

</div>

@bramo That’s great! No, the microbot service is just a simple test to verify that it is working, you can kill it.

---

<div class="post-metadata">

### Author: ![3kWTio](https://avatars.discourse-cdn.com/v4/letter/3/3ab097/32.png) [@3kWTio](https://discuss.kubernetes.io/u/3kWTio)
#### Post date: [December 21, 2022, 7:14pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/7 "2022-12-21T19:14:39Z")

</div>

Been trying to get this to work for days without success.

Today i reset my microk8s-cluster with `snap remove microk8s --purge`, then start with a fresh

`sudo snap install microk8s --classic --channel=1.26/stable`

and go on with the following commands:

```auto
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
su - $USER
microk8s status --wait-ready
sudo snap alias microk8s.kubectl kubectl
sudo snap alias microk8s.helm3 helm
helm repo add stable https://charts.helm.sh/stable
helm repo update
microk8s add-node 
    #then add the 2 other nodes with
microk8s join ....
microk8s enable dns:192.168.1.1,5.1.66.255 # CoreDNS
microk8s enable host-access # Allow Pods connecting to Host services smoothly
microk8s enable ingress # Ingress controller for external access
microk8s enable dashboard # The Kubernetes dashboard
microk8s enable metallb:'192.168.1.10-192.168.1.10,192.168.1.240-192.168.1.253'
microk8s enable cert-manager
kubectl apply -f clusterissuer.yml

```

Now after a few unsuccesful trials with my own clusterissuer and kuard as app i tried your microbot example from the top with the same result:

```auto
default 1s Normal ScalingReplicaSet deployment/microbot Scaled up replica set microbot-b778687f9 to 3
default 1s Normal SuccessfulCreate replicaset/microbot-b778687f9 Created pod: microbot-b778687f9-5jnvm
default 1s Normal SuccessfulCreate replicaset/microbot-b778687f9 Created pod: microbot-b778687f9-g68kh
default 1s Normal SuccessfulCreate replicaset/microbot-b778687f9 Created pod: microbot-b778687f9-dslhz
default 0s Normal Scheduled pod/microbot-b778687f9-g68kh Successfully assigned default/microbot-b778687f9-g68kh to cluster-wf-technik01
default 0s Normal Scheduled pod/microbot-b778687f9-5jnvm Successfully assigned default/microbot-b778687f9-5jnvm to cluster-wf-essen01
default 0s Normal Scheduled pod/microbot-b778687f9-dslhz Successfully assigned default/microbot-b778687f9-dslhz to cluster-bs-dg01
default 0s Normal Pulling pod/microbot-b778687f9-g68kh Pulling image "cdkbot/microbot:1"
default 0s Normal Pulling pod/microbot-b778687f9-5jnvm Pulling image "cdkbot/microbot:1"
default 0s Normal Pulling pod/microbot-b778687f9-dslhz Pulling image "cdkbot/microbot:1"
default 0s Normal Pulled pod/microbot-b778687f9-g68kh Successfully pulled image "cdkbot/microbot:1" in 6.928614238s (6.928619642s including waiting)
default 0s Normal Created pod/microbot-b778687f9-g68kh Created container microbot
default 1s Normal Started pod/microbot-b778687f9-g68kh Started container microbot
default 0s Normal Pulled pod/microbot-b778687f9-dslhz Successfully pulled image "cdkbot/microbot:1" in 45.021152957s (45.021157699s including waiting)
default 0s Normal Created pod/microbot-b778687f9-dslhz Created container microbot
default 0s Normal Started pod/microbot-b778687f9-dslhz Started container microbot
default 0s Normal Pulled pod/microbot-b778687f9-5jnvm Successfully pulled image "cdkbot/microbot:1" in 47.301921331s (47.301925255s including waiting)
default 0s Normal Created pod/microbot-b778687f9-5jnvm Created container microbot
default 0s Normal Started pod/microbot-b778687f9-5jnvm Started container microbot
ingress 0s Normal RELOAD pod/nginx-ingress-microk8s-controller-l46t4 NGINX reload triggered due to a change in configuration
ingress 0s Normal RELOAD pod/nginx-ingress-microk8s-controller-pxtlb NGINX reload triggered due to a change in configuration
ingress 1s Normal RELOAD pod/nginx-ingress-microk8s-controller-dnqkv NGINX reload triggered due to a change in configuration
default 0s Normal Sync ingress/microbot-ingress Scheduled for sync
default 0s Normal Sync ingress/microbot-ingress Scheduled for sync
ingress 0s Normal RELOAD pod/nginx-ingress-microk8s-controller-pxtlb NGINX reload triggered due to a change in configuration
ingress 0s Normal RELOAD pod/nginx-ingress-microk8s-controller-l46t4 NGINX reload triggered due to a change in configuration
default 0s Normal CreateCertificate ingress/microbot-ingress Successfully created Certificate "microbot-ingress-tls"
default 0s Normal Sync ingress/microbot-ingress Scheduled for sync
ingress 0s Normal RELOAD pod/nginx-ingress-microk8s-controller-dnqkv NGINX reload triggered due to a change in configuration
default 0s Normal Issuing certificate/microbot-ingress-tls Issuing certificate as Secret does not exist
default 0s Normal Generated certificate/microbot-ingress-tls Stored new private key in temporary Secret resource "microbot-ingress-tls-p9j5v"
default 0s Normal Requested certificate/microbot-ingress-tls Created new CertificateRequest resource "microbot-ingress-tls-tsmj9"
default 0s Normal cert-manager.io certificaterequest/microbot-ingress-tls-tsmj9 Certificate request has been approved by cert-manager.io
default 1s Normal OrderCreated certificaterequest/microbot-ingress-tls-tsmj9 Created Order resource default/microbot-ingress-tls-tsmj9-3484024173
default 1s Normal OrderPending certificaterequest/microbot-ingress-tls-tsmj9 Waiting on certificate issuance from order default/microbot-ingress-tls-tsmj9-3484024173: ""
default 0s Normal Sync ingress/microbot-ingress Scheduled for sync
default 0s Normal Sync ingress/microbot-ingress Scheduled for sync
default 0s Normal Sync ingress/microbot-ingress Scheduled for sync
default 1s Normal Created order/microbot-ingress-tls-tsmj9-3484024173 Created Challenge resource "microbot-ingress-tls-tsmj9-3484024173-1852125348" for domain "dahoam.13mail.de"
default 0s Normal Started challenge/microbot-ingress-tls-tsmj9-3484024173-1852125348 Challenge scheduled for processing
default 0s Normal Sync ingress/cm-acme-http-solver-4c9x2 Scheduled for sync
default 0s Normal Presented challenge/microbot-ingress-tls-tsmj9-3484024173-1852125348 Presented challenge using HTTP-01 challenge mechanism
default 0s Normal Sync ingress/cm-acme-http-solver-4c9x2 Scheduled for sync
default 0s Normal Sync ingress/cm-acme-http-solver-5tk2v Scheduled for sync
default 0s Normal Sync ingress/cm-acme-http-solver-5tk2v Scheduled for sync
default 0s Normal Scheduled pod/cm-acme-http-solver-9df5d Successfully assigned default/cm-acme-http-solver-9df5d to cluster-bs-dg01
default 0s Normal Sync ingress/cm-acme-http-solver-4c9x2 Scheduled for sync
default 0s Normal Sync ingress/cm-acme-http-solver-5tk2v Scheduled for sync
default 0s Normal Sync ingress/cm-acme-http-solver-hg7j9 Scheduled for sync
default 0s Normal Sync ingress/cm-acme-http-solver-hg7j9 Scheduled for sync
default 0s Normal Sync ingress/cm-acme-http-solver-hg7j9 Scheduled for sync
default 0s Normal Pulling pod/cm-acme-http-solver-9df5d Pulling image "quay.io/jetstack/cert-manager-acmesolver:v1.8.0"
ingress 1s Normal RELOAD pod/nginx-ingress-microk8s-controller-pxtlb NGINX reload triggered due to a change in configuration
ingress 1s Normal RELOAD pod/nginx-ingress-microk8s-controller-l46t4 NGINX reload triggered due to a change in configuration
ingress 1s Normal RELOAD pod/nginx-ingress-microk8s-controller-dnqkv NGINX reload triggered due to a change in configuration
default 0s Normal Pulled pod/cm-acme-http-solver-9df5d Successfully pulled image "quay.io/jetstack/cert-manager-acmesolver:v1.8.0" in 18.433090934s (18.433100365s including waiting)
default 0s Normal Created pod/cm-acme-http-solver-9df5d Created container acmesolver
default 0s Normal Started pod/cm-acme-http-solver-9df5d Started container acmesolver
default 0s Normal Sync ingress/cm-acme-http-solver-hg7j9 Scheduled for sync
default 0s Normal Sync ingress/cm-acme-http-solver-hg7j9 Scheduled for sync
default 0s Normal Sync ingress/cm-acme-http-solver-hg7j9 Scheduled for sync
default 0s Normal Killing pod/cm-acme-http-solver-9df5d Stopping container acmesolver
ingress 0s Normal RELOAD pod/nginx-ingress-microk8s-controller-l46t4 NGINX reload triggered due to a change in configuration
ingress 0s Normal RELOAD pod/nginx-ingress-microk8s-controller-pxtlb NGINX reload triggered due to a change in configuration
default 0s Warning OrderFailed certificaterequest/microbot-ingress-tls-tsmj9 Failed to wait for order resource "microbot-ingress-tls-tsmj9-3484024173" to become ready: order is in "invalid" state:
default 1s Warning Failed certificate/microbot-ingress-tls The certificate request has failed to complete and will be retried: Failed to wait for order resource "microbot-ingress-tls-tsmj9-3484024173" to become ready: order is in "invalid" state:
ingress 1s Normal RELOAD pod/nginx-ingress-microk8s-controller-dnqkv NGINX reload triggered due to a change in configuration
ingress 1s Normal RELOAD pod/nginx-ingress-microk8s-controller-pxtlb NGINX reload triggered due to a change in configuration
ingress 1s Normal RELOAD pod/nginx-ingress-microk8s-controller-l46t4 NGINX reload triggered due to a change in configuration

```

```auto
kubectl describe certificaterequest microbot-ingress-tls-tsmj9
Name: microbot-ingress-tls-tsmj9
Namespace: default
Labels: <none>
Annotations: cert-manager.io/certificate-name: microbot-ingress-tls
              cert-manager.io/certificate-revision: 1
              cert-manager.io/private-key-secret-name: microbot-ingress-tls-p9j5v
API Version: cert-manager.io/v1
Kind: CertificateRequest
Metadata:
  Creation Timestamp: 2022-12-21T18:52:59Z
  Generate Name: microbot-ingress-tls-
  Generation: 1
  Managed Fields:
    API Version: cert-manager.io/v1
    Fields Type: FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
          .:
          k:{"type":"Approved"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
            f:type:
    Manager: cert-manager-certificaterequests-approver
    Operation: Update
    Subresource: status
    Time: 2022-12-21T18:52:59Z
    API Version: cert-manager.io/v1
    Fields Type: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:cert-manager.io/certificate-name:
          f:cert-manager.io/certificate-revision:
          f:cert-manager.io/private-key-secret-name:
        f:generateName:
        f:ownerReferences:
          .:
          k:{"uid":"64aa8ec9-ce23-4a26-9637-d400783d1c3b"}:
      f:spec:
        .:
        f:issuerRef:
          .:
          f:group:
          f:kind:
          f:name:
        f:request:
        f:usages:
    Manager: cert-manager-certificates-request-manager
    Operation: Update
    Time: 2022-12-21T18:52:59Z
    API Version: cert-manager.io/v1
    Fields Type: FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
          k:{"type":"Ready"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
            f:type:
        f:failureTime:
    Manager: cert-manager-certificaterequests-issuer-acme
    Operation: Update
    Subresource: status
    Time: 2022-12-21T18:54:09Z
  Owner References:
    API Version: cert-manager.io/v1
    Block Owner Deletion: true
    Controller: true
    Kind: Certificate
    Name: microbot-ingress-tls
    UID: 64aa8ec9-ce23-4a26-9637-d400783d1c3b
  Resource Version: 27090
  UID: 783f9bab-b1bf-4a59-8ce4-fcd5c4ccb88d
Spec:
  Extra:
    authentication.kubernetes.io/pod-name:
      cert-manager-69c6cb69f9-mwldw
    authentication.kubernetes.io/pod-uid:
      945ccb41-a5ae-4aaf-9535-f16dcb751809
  Groups:
    system:serviceaccounts
    system:serviceaccounts:cert-manager
    system:authenticated
  Issuer Ref:
    Group: cert-manager.io
    Kind: ClusterIssuer
    Name: lets-encrypt
  Request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ2dEQ0NBV2dDQVFBd0FEQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUtwaAp1eHpnaUk2blNUQUZFWEVYeVpxYit6eGpIY1c1ZWU0WDdKaHlXNThwMk1qZEZCQmVNNmg1WmQ2YlFTTVdCVFZKClowVUM3ZGk1Mms0QS9HMEl1OGpKbDR6cVRRS0pveVVuc2hwL3dhWkNtd1lrMUxmZ1czMmdvTmcySE5OR0wyRm8Kc1JDMHNadHFObFZEdXdtUUI2Y2dpRWhRR0ZwekZ0VlNYVjFPaEJ4R1JpdXo4TWtaRnVDeDRPZk9WVmJ1bmYyQQpBUWkwWHBGRXdWODdGMnk3aThLMldkVmc2cXY3dHhVaWduTVVSWDRmZVQ2bzF5R0hwdWZlRUJEL05oNmNXVk9SCndYb1ptS0plR09acU1jd2dLdE5tSlpwdzVGS2lMODlFMEtnY1lDUmZCTW9nalBXWkplSFVXOHgvZWplSEJDSG4KRldYUXhTaGZXdEVHZHlyVVdDTUNBd0VBQWFBN01Ea0dDU3FHU0liM0RRRUpEakVzTUNvd0d3WURWUjBSQkJRdwpFb0lRWkdGb2IyRnRMakV6YldGcGJDNWtaVEFMQmdOVkhROEVCQU1DQmFBd0RRWUpLb1pJaHZjTkFRRUxCUUFECmdnRUJBS1lGR05YVEwyM1RaMmliNCs3NDZIV3lVV1h4bW1SK01CQTNYaVJkOXlWNVY4NXp5b253bHdNdDExT0wKTTZoakR4VEVVZnpMcnJOaFgxcHpCaERIdDlSdDU2eWUxanppb3ZKcTVzTDQ0NktpUmpkNmI3dUNpT3Y0T0EwcAp0ZTVZUVIrSDd2cGFqMnBvY0tuNEtvUzNHTVZ5bjV0MnU4bkpyZDhHbFI1ZTBhSi9UQVhqcDhJaGVTbjFra3FNCmhieFdZbTJqM2JtTUZkMmpVblhpOC80ZTA5NVZEblpaVXV0azYxSzByZWVIUUVZaGZ5TkVna25Vb3BqS3JjaHAKVVdKbzU0NHlXYWsrcVFpZGNhazZDRlNPMEZOcGlpbWFWeGd1ZFFGU3EyZW9oUklnR0k5SndZVm5FRXFpYlU0cgpnSFBndFBXc29MSTJ0cGlqNzNWejZUbHVvdms9Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=
  UID: 1e592ac4-61ca-435f-a6bf-b46b5389156a
  Usages:
    digital signature
    key encipherment
  Username: system:serviceaccount:cert-manager:cert-manager
Status:
  Conditions:
    Last Transition Time: 2022-12-21T18:52:59Z
    Message: Certificate request has been approved by cert-manager.io
    Reason: cert-manager.io
    Status: True
    Type: Approved
    Last Transition Time: 2022-12-21T18:52:59Z
    Message: Failed to wait for order resource "microbot-ingress-tls-tsmj9-3484024173" to become ready: order is in "invalid" state: 
    Reason: Failed
    Status: False
    Type: Ready
  Failure Time: 2022-12-21T18:54:09Z
Events:
  Type Reason Age From Message
  ---- ------ ---- ---- -------
  Normal cert-manager.io 4m16s cert-manager-certificaterequests-approver Certificate request has been approved by cert-manager.io
  Normal OrderCreated 4m16s cert-manager-certificaterequests-issuer-acme Created Order resource default/microbot-ingress-tls-tsmj9-3484024173
  Normal OrderPending 4m16s cert-manager-certificaterequests-issuer-acme Waiting on certificate issuance from order default/microbot-ingress-tls-tsmj9-3484024173: ""
  Warning OrderFailed 3m6s cert-manager-certificaterequests-issuer-acme Failed to wait for order resource "microbot-ingress-tls-tsmj9-3484024173" to become ready: order is in "invalid" state:

```

```auto
kubectl describe certificate microbot-ingress-tls
Name: microbot-ingress-tls
Namespace: default
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1
Kind: Certificate
Metadata:
  Creation Timestamp: 2022-12-21T18:52:58Z
  Generation: 1
  Managed Fields:
    API Version: cert-manager.io/v1
    Fields Type: FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
          k:{"type":"Ready"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:observedGeneration:
            f:reason:
            f:status:
            f:type:
    Manager: cert-manager-certificates-readiness
    Operation: Update
    Subresource: status
    Time: 2022-12-21T18:52:58Z
    API Version: cert-manager.io/v1
    Fields Type: FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
          .:
          k:{"type":"Issuing"}:
            .:
            f:observedGeneration:
            f:type:
    Manager: cert-manager-certificates-trigger
    Operation: Update
    Subresource: status
    Time: 2022-12-21T18:52:58Z
    API Version: cert-manager.io/v1
    Fields Type: FieldsV1
    fieldsV1:
      f:metadata:
        f:ownerReferences:
          .:
          k:{"uid":"18e12242-3f02-4277-9814-3450face7ba8"}:
      f:spec:
        .:
        f:dnsNames:
        f:issuerRef:
          .:
          f:group:
          f:kind:
          f:name:
        f:secretName:
        f:usages:
    Manager: cert-manager-ingress-shim
    Operation: Update
    Time: 2022-12-21T18:52:58Z
    API Version: cert-manager.io/v1
    Fields Type: FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
          k:{"type":"Issuing"}:
            f:lastTransitionTime:
            f:message:
            f:reason:
            f:status:
        f:failedIssuanceAttempts:
        f:lastFailureTime:
    Manager: cert-manager-certificates-issuing
    Operation: Update
    Subresource: status
    Time: 2022-12-21T18:54:09Z
  Owner References:
    API Version: networking.k8s.io/v1
    Block Owner Deletion: true
    Controller: true
    Kind: Ingress
    Name: microbot-ingress
    UID: 18e12242-3f02-4277-9814-3450face7ba8
  Resource Version: 27100
  UID: 64aa8ec9-ce23-4a26-9637-d400783d1c3b
Spec:
  Dns Names:
    something.mydomain.com
  Issuer Ref:
    Group: cert-manager.io
    Kind: ClusterIssuer
    Name: lets-encrypt
  Secret Name: microbot-ingress-tls
  Usages:
    digital signature
    key encipherment
Status:
  Conditions:
    Last Transition Time: 2022-12-21T18:54:09Z
    Message: The certificate request has failed to complete and will be retried: Failed to wait for order resource "microbot-ingress-tls-tsmj9-3484024173" to become ready: order is in "invalid" state: 
    Observed Generation: 1
    Reason: Failed
    Status: False
    Type: Issuing
    Last Transition Time: 2022-12-21T18:52:58Z
    Message: Issuing certificate as Secret does not exist
    Observed Generation: 1
    Reason: DoesNotExist
    Status: False
    Type: Ready
  Failed Issuance Attempts: 1
  Last Failure Time: 2022-12-21T18:54:09Z
Events:
  Type Reason Age From Message
  ---- ------ ---- ---- -------
  Normal Issuing 4m53s cert-manager-certificates-trigger Issuing certificate as Secret does not exist
  Normal Generated 4m52s cert-manager-certificates-key-manager Stored new private key in temporary Secret resource "microbot-ingress-tls-p9j5v"
  Normal Requested 4m52s cert-manager-certificates-request-manager Created new CertificateRequest resource "microbot-ingress-tls-tsmj9"
  Warning Failed 3m42s cert-manager-certificates-issuing The certificate request has failed to complete and will be retried: Failed to wait for order resource "microbot-ingress-tls-tsmj9-3484024173" to become ready: order is in "invalid" state:

```

I am at my wit’s end and am grateful for any help

---

<div class="post-metadata">

### Author: ![Stanislav\_Trifan](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/stanislav_trifan/32/12690_2.png) [@Stanislav\_Trifan](https://discuss.kubernetes.io/u/Stanislav_Trifan)
#### Post date: [June 9, 2023, 1:32pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/8 "2023-06-09T13:32:35Z")

</div>

During the

> [@evilnick](#):
>
> `microk8s kubectl apply -f - <<EOF...`

getting the next error:  
`Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": context deadline exceeded `

---

<div class="post-metadata">

### Author: ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)
#### Post date: [June 9, 2023, 2:02pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/9 "2023-06-09T14:02:21Z")

</div>

@Stanislav_Trifan sorry it isn’t working for you. That error indicates the pod isn’t being created in the expected timeframe, which could be for a number of reasons. Could you paste here the exact YAML of the resource you tried to create?

---

<div class="post-metadata">

### Author: ![Stanislav\_Trifan](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/stanislav_trifan/32/12690_2.png) [@Stanislav\_Trifan](https://discuss.kubernetes.io/u/Stanislav_Trifan)
#### Post date: [June 9, 2023, 5:05pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/10 "2023-06-09T17:05:32Z")

</div>

@evilnick this is the command I run (on microk8s 1.27.2 rev 5372):

```auto

stanislav@dev:~/workspace/homelab/kubecluster$ kubectl apply -f - <<EOF
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
 name: lets-encrypt
spec:
 acme:
   email: _edited_@gmail.com
   server: https://acme-v02.api.letsencrypt.org/directory
   privateKeySecretRef:
     # Secret resource that will be used to store the account's private key.
     name: lets-encrypt-priviate-key
   # Add a single challenge solver, HTTP01 using nginx
   solvers:
   - http01:
       ingress:
         class: public
EOF

Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": context deadline exceeded

```

---

<div class="post-metadata">

### Author: ![kjackal](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/kjackal/32/1750_2.png) [@kjackal](https://discuss.kubernetes.io/u/kjackal)
#### Post date: [June 13, 2023, 2:30pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/11 "2023-06-13T14:30:23Z")

</div>

@Stanislav_Trifan since I cannot reproduce the issue, could you please file an issue on [Issues · canonical/microk8s · GitHub](https://github.com/canonical/microk8s/issues) and attach a `microk8s inspect` tarball?

---

<div class="post-metadata">

### Author: ![o0th](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/o0th/32/15105_2.png) [@o0th](https://discuss.kubernetes.io/u/o0th)
#### Post date: [May 30, 2024, 5:03pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/12 "2024-05-30T17:03:04Z")

</div>

> [@evilnick](#):
>
> After a while, Cert-Manager will automatically request a certificate from Let’s Encrypt, populate the `microk8s-ingress-tls` with it and configure the ingress. Finally, you should be able to access your service at its fully qualified domain.

Intead of `microk8s-ingress-tls` it should be `microbot-ingress-tls`

---

<div class="post-metadata">

### Author: ![jjg](https://avatars.discourse-cdn.com/v4/letter/j/58f4c7/32.png) [@jjg](https://discuss.kubernetes.io/u/jjg)
#### Post date: [October 8, 2024, 2:12pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/13 "2024-10-08T14:12:58Z")

</div>

There’s a typo in the original instructions—lets-encrypt-priviate-key should be lets-encrypt-private-key.

I know it does not impact anything.

---

<div class="post-metadata">

### Author: ![jjg](https://avatars.discourse-cdn.com/v4/letter/j/58f4c7/32.png) [@jjg](https://discuss.kubernetes.io/u/jjg)
#### Post date: [October 9, 2024, 2:21pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/14 "2024-10-09T14:21:26Z")

</div>

Solver’s gets wrong class following instructions, need to manually set.

Here are my logs and solution from ChatGPT.

followed instructions on [MicroK8s - Addon: cert-manager](https://microk8s.io/docs/addon-cert-manager)  
but ended up with solvers with class .

I enabled all the required packages, and even rebooted the server.

my ClusterIssuer is as follows:  
apiVersion: [cert-manager.io/v1](http://cert-manager.io/v1)  
kind: ClusterIssuer  
metadata:  
name: lets-encrypt # Changed to match MicroK8s instructions  
spec:  
acme:  
email: [john\_grabner@hotmail.com](mailto:john_grabner@hotmail.com) # Your email address  
server: [https://acme-v02.api.letsencrypt.org/directory](https://acme-v02.api.letsencrypt.org/directory)  
privateKeySecretRef:  
name: lets-encrypt-private-key # Corrected and matched name  
solvers:  
- http01:  
ingress:  
class: public

\*\*\* later you will see solver do not get class public but class . Not sure if this is an issue, but looks odd.

it is ready as can be seen here:

```auto
icrok8s kubectl get clusterissuer -o wide
NAME READY STATUS AGE
lets-encrypt True The ACME account was registered with the ACME server 23h

```

my service is running as can be seen here:

```auto
microk8s kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/cm-acme-http-solver-2dfxw 1/1 Running 2 (7m21s ago) 20h
pod/cm-acme-http-solver-9p244 1/1 Running 2 (7m21s ago) 20h
pod/express-deployment-7bc8645c58-kn6f4 1/1 Running 2 (7m21s ago) 21h
pod/mysql-deployment-59b74b59f9-4t8x8 2/2 Running 4 (7m21s ago) 41h

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/cm-acme-http-solver-cf2sq NodePort 10.152.183.29 <none> 8089:31437/TCP 20h
service/cm-acme-http-solver-dffrw NodePort 10.152.183.222 <none> 8089:30921/TCP 20h
service/express-service ClusterIP 10.152.183.208 <none> 3000/TCP,55001/TCP 21h
service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 43h
service/mysql-service NodePort 10.152.183.189 <none> 3306:30306/TCP 41h

```

my ingress is as follows:

```auto
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-before-ttl
  annotations:
    cert-manager.io/cluster-issuer: "lets-encrypt"
    nginx.ingress.kubernetes.io/affinity: "cookie"
    nginx.ingress.kubernetes.io/proxy-body-size: 16m
spec:
  tls:
    - hosts:
        - ancient-script.org
        - www.ancient-script.org
      secretName: ancient-script-org-crt-secret
  rules:
    - host: ancient-script.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: express-service
                port:
                  number: 3000
    - host: www.ancient-script.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: express-service
                port:
                  number: 3000

```

k describe ingress

```auto
Name: cm-acme-http-solver-rzl2p
Labels: acme.cert-manager.io/http-domain=1683425454
                  acme.cert-manager.io/http-token=829886008
                  acme.cert-manager.io/http01-solver=true
Namespace: default
Address: 127.0.0.1
Ingress Class: <none>
Default backend: <default>
Rules:
  Host Path Backends
  ---- ---- --------
  www.ancient-script.org  
                          /.well-known/acme-challenge/6c1PhtRjg3Dz-MLKeDXe1mES-0YsZjtzN5NON17zwks cm-acme-http-solver-cf2sq:8089 (10.1.96.84:8089)
Annotations: kubernetes.io/ingress.class: public
                          nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0,::/0
Events: <none>

Name: cm-acme-http-solver-w4bdc
Labels: acme.cert-manager.io/http-domain=1134561051
                  acme.cert-manager.io/http-token=1348800303
                  acme.cert-manager.io/http01-solver=true
Namespace: default
Address: 127.0.0.1
Ingress Class: <none>
Default backend: <default>
Rules:
  Host Path Backends
  ---- ---- --------
  ancient-script.org  
                      /.well-known/acme-challenge/WkTcEHzZlXlUo3mtd4E2KKLzhSzynnz5bh5e7N7Yw_Y cm-acme-http-solver-dffrw:8089 (10.1.96.93:8089)
Annotations: kubernetes.io/ingress.class: public
                      nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0,::/0
Events: <none>

Name: ingress-before-ttl
Labels: <none>
Namespace: default
Address: 127.0.0.1
Ingress Class: public
Default backend: <default>
TLS:
  ancient-script-org-crt-secret terminates ancient-script.org,www.ancient-script.org
Rules:
  Host Path Backends
  ---- ---- --------
  ancient-script.org      
                          / express-service:3000 (10.1.96.96:3000)
  www.ancient-script.org  
                          / express-service:3000 (10.1.96.96:3000)
Annotations: cert-manager.io/cluster-issuer: lets-encrypt
                          nginx.ingress.kubernetes.io/affinity: cookie
                          nginx.ingress.kubernetes.io/proxy-body-size: 16m
Events: <none>

```

k describe certificate

```auto
Name: ancient-script-org-crt-secret
Namespace: default
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1
Kind: Certificate
Metadata:
  Creation Timestamp: 2024-10-08T16:43:08Z
  Generation: 1
  Owner References:
    API Version: networking.k8s.io/v1
    Block Owner Deletion: true
    Controller: true
    Kind: Ingress
    Name: ingress-before-ttl
    UID: f065e265-d6ab-46a7-a5bd-507d0226329d
  Resource Version: 237082
  UID: 69adfc17-9572-4f90-a8b2-db0deaf6606b
Spec:
  Dns Names:
    ancient-script.org
    www.ancient-script.org
  Issuer Ref:
    Group: cert-manager.io
    Kind: ClusterIssuer
    Name: lets-encrypt
  Secret Name: ancient-script-org-crt-secret
  Usages:
    digital signature
    key encipherment
Status:
  Conditions:
    Last Transition Time: 2024-10-08T16:43:08Z
    Message: Issuing certificate as Secret does not exist
    Observed Generation: 1
    Reason: DoesNotExist
    Status: False
    Type: Ready
    Last Transition Time: 2024-10-08T16:43:08Z
    Message: Issuing certificate as Secret does not exist
    Observed Generation: 1
    Reason: DoesNotExist
    Status: True
    Type: Issuing
  Next Private Key Secret Name: ancient-script-org-crt-secret-4k8wr
Events: <none>

```

k -n cert-manager logs cert-manager-cainjector-dc95f9d66-m449x

```auto
I1009 13:28:16.672686 1 start.go:126] "starting" version="v1.8.0" revision="e466a521bc5455def8c224599c6edcd37e86410c"
I1009 13:28:26.691560 1 leaderelection.go:248] attempting to acquire leader lease kube-system/cert-manager-cainjector-leader-election...
I1009 13:29:47.067146 1 leaderelection.go:258] successfully acquired lease kube-system/cert-manager-cainjector-leader-election
I1009 13:29:47.067324 1 recorder.go:103] cert-manager/events "msg"="Normal" "message"="cert-manager-cainjector-dc95f9d66-m449x_de1669f2-a803-4069-ac81-8f2c061ecf44 became leader" "object"={"kind":"Lease","namespace":"kube-system","name":"cert-manager-cainjector-leader-election","uid":"34db52f2-e5dc-4643-b33c-911f30212a0c","apiVersion":"coordination.k8s.io/v1","resourceVersion":"241297"} "reason"="LeaderElection"
I1009 13:29:47.168218 1 controller.go:178] cert-manager/certificate/mutatingwebhookconfiguration/controller/controller-for-certificate-mutatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.MutatingWebhookConfiguration=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} []}) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168226 1 controller.go:178] cert-manager/secret/customresourcedefinition/controller/controller-for-secret-customresourcedefinition "msg"="Starting EventSource" "source"="&{{%!s(*v1.CustomResourceDefinition=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} { { [] []} [] <nil> false} {[] { [] []} []}}) %!s(*cache.informerCache=&{0xc000720900}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168251 1 controller.go:178] cert-manager/certificate/mutatingwebhookconfiguration/controller/controller-for-certificate-mutatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.Certificate=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} {<nil> <nil> <nil> [] [] [] [] <nil> <nil> { } false [] <nil> <nil> <nil> []} {[] <nil> <nil> <nil> <nil> <nil> <nil> <nil>}}) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168258 1 controller.go:178] cert-manager/secret/customresourcedefinition/controller/controller-for-secret-customresourcedefinition "msg"="Starting EventSource" "source"="&{{%!s(*v1.Secret=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} <nil> map[] map[] }) %!s(*cache.informerCache=&{0xc000720900}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168271 1 controller.go:186] cert-manager/secret/customresourcedefinition/controller/controller-for-secret-customresourcedefinition "msg"="Starting Controller"  
I1009 13:29:47.168273 1 controller.go:178] cert-manager/certificate/mutatingwebhookconfiguration/controller/controller-for-certificate-mutatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.Secret=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} <nil> map[] map[] }) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168291 1 controller.go:186] cert-manager/certificate/mutatingwebhookconfiguration/controller/controller-for-certificate-mutatingwebhookconfiguration "msg"="Starting Controller"  
I1009 13:29:47.168321 1 controller.go:178] cert-manager/certificate/apiservice/controller/controller-for-certificate-apiservice "msg"="Starting EventSource" "source"="&{{%!s(*v1.APIService=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} {<nil> false [] 0 0} {[]}}) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168352 1 controller.go:178] cert-manager/certificate/apiservice/controller/controller-for-certificate-apiservice "msg"="Starting EventSource" "source"="&{{%!s(*v1.Certificate=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} {<nil> <nil> <nil> [] [] [] [] <nil> <nil> { } false [] <nil> <nil> <nil> []} {[] <nil> <nil> <nil> <nil> <nil> <nil> <nil>}}) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168370 1 controller.go:178] cert-manager/certificate/apiservice/controller/controller-for-certificate-apiservice "msg"="Starting EventSource" "source"="&{{%!s(*v1.Secret=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} <nil> map[] map[] }) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168377 1 controller.go:186] cert-manager/certificate/apiservice/controller/controller-for-certificate-apiservice "msg"="Starting Controller"  
I1009 13:29:47.168418 1 controller.go:178] cert-manager/secret/validatingwebhookconfiguration/controller/controller-for-secret-validatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.ValidatingWebhookConfiguration=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} []}) %!s(*cache.informerCache=&{0xc000720900}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168446 1 controller.go:178] cert-manager/secret/validatingwebhookconfiguration/controller/controller-for-secret-validatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.Secret=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} <nil> map[] map[] }) %!s(*cache.informerCache=&{0xc000720900}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168459 1 controller.go:186] cert-manager/secret/validatingwebhookconfiguration/controller/controller-for-secret-validatingwebhookconfiguration "msg"="Starting Controller"  
I1009 13:29:47.168580 1 controller.go:178] cert-manager/secret/mutatingwebhookconfiguration/controller/controller-for-secret-mutatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.MutatingWebhookConfiguration=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} []}) %!s(*cache.informerCache=&{0xc000720900}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168611 1 controller.go:178] cert-manager/secret/apiservice/controller/controller-for-secret-apiservice "msg"="Starting EventSource" "source"="&{{%!s(*v1.APIService=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} {<nil> false [] 0 0} {[]}}) %!s(*cache.informerCache=&{0xc000720900}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168642 1 controller.go:178] cert-manager/certificate/validatingwebhookconfiguration/controller/controller-for-certificate-validatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.ValidatingWebhookConfiguration=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} []}) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168646 1 controller.go:178] cert-manager/secret/apiservice/controller/controller-for-secret-apiservice "msg"="Starting EventSource" "source"="&{{%!s(*v1.Secret=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} <nil> map[] map[] }) %!s(*cache.informerCache=&{0xc000720900}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168661 1 controller.go:178] cert-manager/certificate/customresourcedefinition/controller/controller-for-certificate-customresourcedefinition "msg"="Starting EventSource" "source"="&{{%!s(*v1.CustomResourceDefinition=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} { { [] []} [] <nil> false} {[] { [] []} []}}) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168669 1 controller.go:178] cert-manager/certificate/validatingwebhookconfiguration/controller/controller-for-certificate-validatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.Certificate=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} {<nil> <nil> <nil> [] [] [] [] <nil> <nil> { } false [] <nil> <nil> <nil> []} {[] <nil> <nil> <nil> <nil> <nil> <nil> <nil>}}) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168667 1 controller.go:186] cert-manager/secret/apiservice/controller/controller-for-secret-apiservice "msg"="Starting Controller"  
I1009 13:29:47.168690 1 controller.go:178] cert-manager/certificate/customresourcedefinition/controller/controller-for-certificate-customresourcedefinition "msg"="Starting EventSource" "source"="&{{%!s(*v1.Certificate=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} {<nil> <nil> <nil> [] [] [] [] <nil> <nil> { } false [] <nil> <nil> <nil> []} {[] <nil> <nil> <nil> <nil> <nil> <nil> <nil>}}) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168643 1 controller.go:178] cert-manager/secret/mutatingwebhookconfiguration/controller/controller-for-secret-mutatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.Secret=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} <nil> map[] map[] }) %!s(*cache.informerCache=&{0xc000720900}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168707 1 controller.go:178] cert-manager/certificate/customresourcedefinition/controller/controller-for-certificate-customresourcedefinition "msg"="Starting EventSource" "source"="&{{%!s(*v1.Secret=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} <nil> map[] map[] }) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168682 1 controller.go:178] cert-manager/certificate/validatingwebhookconfiguration/controller/controller-for-certificate-validatingwebhookconfiguration "msg"="Starting EventSource" "source"="&{{%!s(*v1.Secret=&{{ } { 0 {{0 0 <nil>}} <nil> <nil> map[] map[] [] [] []} <nil> map[] map[] }) %!s(*cache.informerCache=&{0xc00050e360}) %!s(chan error=<nil>) %!s(func()=<nil>)}}"
I1009 13:29:47.168722 1 controller.go:186] cert-manager/certificate/customresourcedefinition/controller/controller-for-certificate-customresourcedefinition "msg"="Starting Controller"  
I1009 13:29:47.168742 1 controller.go:186] cert-manager/secret/mutatingwebhookconfiguration/controller/controller-for-secret-mutatingwebhookconfiguration "msg"="Starting Controller"  
I1009 13:29:47.168765 1 controller.go:186] cert-manager/certificate/validatingwebhookconfiguration/controller/controller-for-certificate-validatingwebhookconfiguration "msg"="Starting Controller"  
I1009 13:29:47.269434 1 controller.go:220] cert-manager/certificate/mutatingwebhookconfiguration/controller/controller-for-certificate-mutatingwebhookconfiguration "msg"="Starting workers" "worker count"=1
I1009 13:29:47.269514 1 controller.go:220] cert-manager/secret/mutatingwebhookconfiguration/controller/controller-for-secret-mutatingwebhookconfiguration "msg"="Starting workers" "worker count"=1
I1009 13:29:47.269541 1 controller.go:220] cert-manager/certificate/validatingwebhookconfiguration/controller/controller-for-certificate-validatingwebhookconfiguration "msg"="Starting workers" "worker count"=1
I1009 13:29:47.269552 1 controller.go:220] cert-manager/secret/validatingwebhookconfiguration/controller/controller-for-secret-validatingwebhookconfiguration "msg"="Starting workers" "worker count"=1
I1009 13:29:47.269552 1 controller.go:220] cert-manager/certificate/customresourcedefinition/controller/controller-for-certificate-customresourcedefinition "msg"="Starting workers" "worker count"=1
I1009 13:29:47.269578 1 controller.go:220] cert-manager/certificate/apiservice/controller/controller-for-certificate-apiservice "msg"="Starting workers" "worker count"=1
I1009 13:29:47.269585 1 controller.go:220] cert-manager/secret/customresourcedefinition/controller/controller-for-secret-customresourcedefinition "msg"="Starting workers" "worker count"=1
I1009 13:29:47.269609 1 controller.go:220] cert-manager/secret/apiservice/controller/controller-for-secret-apiservice "msg"="Starting workers" "worker count"=1
I1009 13:29:47.274745 1 controller.go:178] cert-manager/secret/mutatingwebhookconfiguration/generic-inject-reconciler "msg"="updated object" "resource_kind"="MutatingWebhookConfiguration" "resource_name"="cert-manager-webhook" "resource_namespace"="" "resource_version"="v1" 
I1009 13:29:47.274946 1 controller.go:178] cert-manager/secret/validatingwebhookconfiguration/generic-inject-reconciler "msg"="updated object" "resource_kind"="ValidatingWebhookConfiguration" "resource_name"="cert-manager-webhook" "resource_namespace"="" "resource_version"="v1" 
I1009 13:29:47.277662 1 controller.go:178] cert-manager/secret/mutatingwebhookconfiguration/generic-inject-reconciler "msg"="updated object" "resource_kind"="MutatingWebhookConfiguration" "resource_name"="cert-manager-webhook" "resource_namespace"="" "resource_version"="v1" 
I1009 13:29:47.320780 1 controller.go:178] cert-manager/secret/customresourcedefinition/generic-inject-reconciler "msg"="updated object" "resource_kind"="CustomResourceDefinition" "resource_name"="issuers.cert-manager.io" "resource_namespace"="" "resource_version"="v1" 
I1009 13:29:47.329779 1 controller.go:178] cert-manager/secret/customresourcedefinition/generic-inject-reconciler "msg"="updated object" "resource_kind"="CustomResourceDefinition" "resource_name"="orders.acme.cert-manager.io" "resource_namespace"="" "resource_version"="v1" 

k -n cert-manager logs cert-manager-d5fcf78bc-xbbjr
I1009 13:28:16.753875 1 start.go:75] cert-manager "msg"="starting controller" "git-commit"="e466a521bc5455def8c224599c6edcd37e86410c" "version"="v1.8.0"
I1009 13:28:16.753958 1 controller.go:242] cert-manager/controller/build-context "msg"="configured acme dns01 nameservers" "nameservers"=["10.152.183.10:53"] 
W1009 13:28:16.754403 1 client_config.go:617] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I1009 13:28:16.759485 1 controller.go:70] cert-manager/controller "msg"="enabled controllers: [certificaterequests-approver certificaterequests-issuer-acme certificaterequests-issuer-ca certificaterequests-issuer-selfsigned certificaterequests-issuer-vault certificaterequests-issuer-venafi certificates-issuing certificates-key-manager certificates-metrics certificates-readiness certificates-request-manager certificates-revision-manager certificates-trigger challenges clusterissuers ingress-shim issuers orders]"  
I1009 13:28:16.759893 1 controller.go:134] cert-manager/controller "msg"="starting leader election"  
I1009 13:28:16.760044 1 controller.go:91] cert-manager/controller "msg"="starting metrics server" "address"={"IP":"::","Port":9402,"Zone":""}
I1009 13:28:16.761026 1 leaderelection.go:248] attempting to acquire leader lease kube-system/cert-manager-controller...
I1009 13:28:16.779671 1 leaderelection.go:258] successfully acquired lease kube-system/cert-manager-controller
I1009 13:28:16.780769 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-issuer-acme" 
I1009 13:28:16.785798 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-issuer-ca" 
I1009 13:28:16.786334 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificates-issuing" 
I1009 13:28:16.786707 1 controller.go:182] cert-manager/controller "msg"="not starting controller as it's disabled" "controller"="certificatesigningrequests-issuer-ca" 
I1009 13:28:16.786728 1 controller.go:182] cert-manager/controller "msg"="not starting controller as it's disabled" "controller"="certificatesigningrequests-issuer-selfsigned" 
I1009 13:28:16.786752 1 controller.go:182] cert-manager/controller "msg"="not starting controller as it's disabled" "controller"="certificatesigningrequests-issuer-vault" 
I1009 13:28:16.786795 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificates-revision-manager" 
I1009 13:28:16.787342 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="orders" 
I1009 13:28:16.787584 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="clusterissuers" 
I1009 13:28:16.788110 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="issuers" 
I1009 13:28:16.788357 1 controller.go:182] cert-manager/controller "msg"="not starting controller as it's disabled" "controller"="gateway-shim" 
I1009 13:28:16.788433 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-approver" 
I1009 13:28:16.792251 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="ingress-shim" 
I1009 13:28:16.792694 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-issuer-selfsigned" 
I1009 13:28:16.793091 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-issuer-vault" 
I1009 13:28:16.793312 1 controller.go:182] cert-manager/controller "msg"="not starting controller as it's disabled" "controller"="certificatesigningrequests-issuer-venafi" 
I1009 13:28:16.793386 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificates-request-manager" 
I1009 13:28:16.898058 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="challenges" 
I1009 13:28:16.898561 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificates-key-manager" 
I1009 13:28:16.898986 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificates-metrics" 
I1009 13:28:16.899422 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificates-readiness" 
I1009 13:28:16.899684 1 controller.go:182] cert-manager/controller "msg"="not starting controller as it's disabled" "controller"="certificatesigningrequests-issuer-acme" 
I1009 13:28:16.899781 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificates-trigger" 
I1009 13:28:16.900181 1 controller.go:205] cert-manager/controller "msg"="starting controller" "controller"="certificaterequests-issuer-venafi" 
I1009 13:28:16.918352 1 util.go:84] cert-manager/controller/certificaterequests-issuer-acme/handleOwnedResource "msg"="owning resource not found in cache" "related_resource_kind"="CertificateRequest" "related_resource_name"="ancient-script-org-crt-secret-cczw9" "related_resource_namespace"="default" "resource_kind"="Order" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274" "resource_namespace"="default" "resource_version"="v1" 
E1009 13:28:16.987548 1 controller.go:166] cert-manager/orders "msg"="re-queuing item due to error processing" "error"="ACME client for issuer not initialised/available" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274" 
I1009 13:28:16.988641 1 setup.go:202] cert-manager/clusterissuers "msg"="skipping re-verifying ACME account as cached registration details look sufficient" "related_resource_kind"="Secret" "related_resource_name"="lets-encrypt-private-key" "related_resource_namespace"="cert-manager" "resource_kind"="ClusterIssuer" "resource_name"="lets-encrypt" "resource_namespace"="" "resource_version"="v1" 
I1009 13:28:16.999452 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-9p244" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:16.999454 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="www.ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-2dfxw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:16.999524 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-dffrw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:16.999546 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="www.ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-cf2sq" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:16.999572 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-w4bdc" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:16.999607 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="www.ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-rzl2p" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:28:27.000640 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://ancient-script.org/.well-known/acme-challenge/WkTcEHzZlXlUo3mtd4E2KKLzhSzynnz5bh5e7N7Yw_Y': Get \"http://ancient-script.org/.well-known/acme-challenge/WkTcEHzZlXlUo3mtd4E2KKLzhSzynnz5bh5e7N7Yw_Y\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" "dnsName"="ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:28:27.000662 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://www.ancient-script.org/.well-known/acme-challenge/6c1PhtRjg3Dz-MLKeDXe1mES-0YsZjtzN5NON17zwks': Get \"http://www.ancient-script.org/.well-known/acme-challenge/6c1PhtRjg3Dz-MLKeDXe1mES-0YsZjtzN5NON17zwks\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" "dnsName"="www.ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:28:31.990103 1 controller.go:166] cert-manager/orders "msg"="re-queuing item due to error processing" "error"="context deadline exceeded" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274" 
E1009 13:28:37.006482 1 controller.go:166] cert-manager/challenges "msg"="re-queuing item due to error processing" "error"="Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274-3325911860" 
E1009 13:28:37.006486 1 controller.go:166] cert-manager/challenges "msg"="re-queuing item due to error processing" "error"="Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274-1378496233" 
I1009 13:28:37.006632 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-9p244" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:37.006702 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-dffrw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:37.006755 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="www.ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-2dfxw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:37.006764 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-w4bdc" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:37.006820 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="www.ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-cf2sq" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:37.006881 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="www.ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-rzl2p" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:28:47.007028 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://www.ancient-script.org/.well-known/acme-challenge/6c1PhtRjg3Dz-MLKeDXe1mES-0YsZjtzN5NON17zwks': Get \"http://www.ancient-script.org/.well-known/acme-challenge/6c1PhtRjg3Dz-MLKeDXe1mES-0YsZjtzN5NON17zwks\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" "dnsName"="www.ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:28:47.007059 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://ancient-script.org/.well-known/acme-challenge/WkTcEHzZlXlUo3mtd4E2KKLzhSzynnz5bh5e7N7Yw_Y': Get \"http://ancient-script.org/.well-known/acme-challenge/WkTcEHzZlXlUo3mtd4E2KKLzhSzynnz5bh5e7N7Yw_Y\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" "dnsName"="ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:28:51.991353 1 controller.go:166] cert-manager/orders "msg"="re-queuing item due to error processing" "error"="context deadline exceeded" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274" 
E1009 13:28:57.011831 1 controller.go:166] cert-manager/challenges "msg"="re-queuing item due to error processing" "error"="Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274-1378496233" 
E1009 13:28:57.011936 1 controller.go:166] cert-manager/challenges "msg"="re-queuing item due to error processing" "error"="Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274-3325911860" 
I1009 13:28:57.012031 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="www.ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-2dfxw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:57.012099 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="www.ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-cf2sq" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:57.012119 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-9p244" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:57.012160 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="www.ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-rzl2p" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:57.012187 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-dffrw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:28:57.012242 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-w4bdc" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:29:07.012540 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://ancient-script.org/.well-known/acme-challenge/WkTcEHzZlXlUo3mtd4E2KKLzhSzynnz5bh5e7N7Yw_Y': Get \"http://ancient-script.org/.well-known/acme-challenge/WkTcEHzZlXlUo3mtd4E2KKLzhSzynnz5bh5e7N7Yw_Y\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" "dnsName"="ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:29:07.012571 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="failed to perform self check GET request 'http://www.ancient-script.org/.well-known/acme-challenge/6c1PhtRjg3Dz-MLKeDXe1mES-0YsZjtzN5NON17zwks': Get \"http://www.ancient-script.org/.well-known/acme-challenge/6c1PhtRjg3Dz-MLKeDXe1mES-0YsZjtzN5NON17zwks\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" "dnsName"="www.ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:29:17.018327 1 controller.go:166] cert-manager/challenges "msg"="re-queuing item due to error processing" "error"="Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274-1378496233" 
E1009 13:29:17.018330 1 controller.go:166] cert-manager/challenges "msg"="re-queuing item due to error processing" "error"="Internal error occurred: failed calling webhook \"webhook.cert-manager.io\": failed to call webhook: Post \"https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s\": context deadline exceeded" "key"="default/ancient-script-org-crt-secret-cczw9-2481331274-3325911860" 
I1009 13:29:17.018531 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-9p244" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:17.018570 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="www.ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-2dfxw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:17.018616 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-dffrw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:17.018665 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="www.ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-cf2sq" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:17.018705 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-w4bdc" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:17.018760 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="www.ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-rzl2p" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:29:17.130108 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="www.ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:29:17.222102 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:27.130744 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="www.ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-2dfxw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:27.130812 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="www.ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-cf2sq" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:27.130863 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="www.ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-rzl2p" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:29:27.143235 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="www.ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:27.222713 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-9p244" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:27.222784 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-dffrw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:27.222849 1 ingress.go:99] cert-manager/challenges/http01/selfCheck/http01/ensureIngress "msg"="found one existing HTTP01 solver ingress" "dnsName"="ancient-script.org" "related_resource_kind"="Ingress" "related_resource_name"="cm-acme-http-solver-w4bdc" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
E1009 13:29:27.233812 1 sync.go:186] cert-manager/challenges "msg"="propagation check failed" "error"="wrong status code '404', expected '200'" "dnsName"="ancient-script.org" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-3325911860" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:37.143602 1 pod.go:59] cert-manager/challenges/http01/selfCheck/http01/ensurePod "msg"="found one existing HTTP01 solver pod" "dnsName"="www.ancient-script.org" "related_resource_kind"="Pod" "related_resource_name"="cm-acme-http-solver-2dfxw" "related_resource_namespace"="default" "related_resource_version"="v1" "resource_kind"="Challenge" "resource_name"="ancient-script-org-crt-secret-cczw9-2481331274-1378496233" "resource_namespace"="default" "resource_version"="v1" "type"="HTTP-01" 
I1009 13:29:37.143678 1 service.go:43] cert-manager/challenges/http01/selfCheck/http01/ensureService "msg"="found one existing HTTP01 solver Service for challenge resource" "dnsName"="www.ancient-script.org" "related_resource_kind"="Service" "related_resource_name"="cm-acme-http-solver-cf2sq" "related_resource_namespace"="default" "related_resource_version"="v1"

```

k get ingress -w

```auto
NAME CLASS HOSTS ADDRESS PORTS AGE
cm-acme-http-solver-rzl2p <none> www.ancient-script.org 127.0.0.1 80 21h
cm-acme-http-solver-w4bdc <none> ancient-script.org 127.0.0.1 80 21h
ingress-before-ttl public ancient-script.org,www.ancient-script.org 127.0.0.1 80, 443 21h

```

\*\*\*\* notice the class for the solvers are while the ingress is public. Not sure if this is an issue, but looks strange.

\*\*\*\* nodejs server does get [ancient-script.org/.well-known/acme-challenge](http://ancient-script.org/.well-known/acme-challenge) and returns a 404 since nodejs does not implement this. Further implying that the solver are not properly hooked in.

I can access my website with [HTTP://ancient-script.org](HTTP://ancient-script.org) from the cloud. i.e. DNS is fine and routing to server is fine.

chatgpt recommended:

Since the HTTP solver ingresses don’t have the public class, you can manually patch the ingresses for the solver to include it.

To patch the solver ingress, use the following command:

bash  
Copy code

```auto
microk8s kubectl patch ingress cm-acme-http-solver-nf7ld -p '{"spec": {"ingressClassName": "public"}}'
microk8s kubectl patch ingress cm-acme-http-solver-s4fh6 -p '{"spec": {"ingressClassName": "public"}}'

```

This will update the solver ingresses to use the correct ingress class, ensuring that traffic for the ACME challenge paths is routed to the solver pods instead of your application.

```auto
k describe certificate
Name: ancient-script-org-crt-secret
Namespace: default
Labels: <none>
Annotations: <none>
API Version: cert-manager.io/v1
Kind: Certificate
Metadata:
  Creation Timestamp: 2024-10-09T14:07:17Z
  Generation: 1
  Owner References:
    API Version: networking.k8s.io/v1
    Block Owner Deletion: true
    Controller: true
    Kind: Ingress
    Name: ingress-before-ttl
    UID: 1f65fec2-e3e2-4bfd-b5ae-f69f4235c3f2
  Resource Version: 248239
  UID: 8de68bd1-fe9f-4824-a0b3-3a7455b56770
Spec:
  Dns Names:
    ancient-script.org
    www.ancient-script.org
  Issuer Ref:
    Group: cert-manager.io
    Kind: ClusterIssuer
    Name: lets-encrypt
  Secret Name: ancient-script-org-crt-secret
  Usages:
    digital signature
    key encipherment
Status:
  Conditions:
    Last Transition Time: 2024-10-09T14:07:17Z
    Message: Issuing certificate as Secret does not exist
    Observed Generation: 1
    Reason: DoesNotExist
    Status: True
    Type: Issuing
    Last Transition Time: 2024-10-09T14:07:17Z
    Message: Issuing certificate as Secret does not exist
    Observed Generation: 1
    Reason: DoesNotExist
    Status: False
    Type: Ready
  Next Private Key Secret Name: ancient-script-org-crt-secret-m727d
Events:
  Type Reason Age From Message
  ---- ------ ---- ---- -------
  Normal Issuing 2m53s cert-manager-certificates-trigger Issuing certificate as Secret does not exist
  Normal Generated 2m53s cert-manager-certificates-key-manager Stored new private key in temporary Secret resource "ancient-script-org-crt-secret-m727d"
  Normal Requested 2m53s cert-manager-certificates-request-manager Created new CertificateRequest resource "ancient-script-org-crt-secret-p6mm4"

```

still have issues, so not full solution.

---

<div class="post-metadata">

### Author: ![jveraa](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/jveraa/32/15920_2.png) [@jveraa](https://discuss.kubernetes.io/u/jveraa)
#### Post date: [October 12, 2024, 12:13pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/15 "2024-10-12T12:13:45Z")

</div>

Hi there, I ended up installing cert-manager myself as it’s pretty easy using argocd + their helm chart.

I did use the example ingress from here, and it contains `pathType: Exact` instead of `pathType: Prefix`. This tripped me up for ~15 minutes as “for some reason the image of the microbot example was not loading”. I’m running the cluster behind another ngnix reversed proxy, which means adding some header forwarding etc., so once I was able to acces the index page on the container I assumed some misconfiguration there was the issue for the image still returning a 404.

Turns out that the Ingress ‘does what it says’, and the example `pathType: Exact` will only match the exact url `https://my-service.example.com` without a subpath, which will serve up the index page as that is configured in the microbot image, so not `https://my-service.example.com/microbot.png`.

I suggest the docs shoud be changed to use `pathType: Prefix`.

---

<div class="post-metadata">

### Author: ![saqrio](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@saqrio](https://discuss.kubernetes.io/u/saqrio)
#### Post date: [October 19, 2024, 7:08am UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/16 "2024-10-19T07:08:20Z")

</div>

{ Original post quoted the entire article, then inserted irrelevant links}

---

<div class="post-metadata">

### Author: ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)
#### Post date: [October 19, 2024, 7:34pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/17 "2024-10-19T19:34:48Z")

</div>

@saqrio Why are you trying to insert this into the instructions?

> [@evilnick](#):
>
> `https://menuland.ph/jollibee-menu-prices-in-philippines/`

---

<div class="post-metadata">

### Author: ![Saikiran\_Biradar](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/saikiran_biradar/32/15755_2.png) [@Saikiran\_Biradar](https://discuss.kubernetes.io/u/Saikiran_Biradar)
#### Post date: [December 2, 2024, 3:41pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/18 "2024-12-02T15:41:08Z")

</div>

@evilnick  
I followed the setps and i got below error

microk8s kubectl get certificates -A  
NAMESPACE NAME READY SECRET AGE  
cert-manager test-50gramx-com False tls-secret 84m  
[ec2-user@ip-172-31-47-8 test]$ microk8s kubectl describe certificates test-50gramx-com -n cert-manager  
Name: test-50gramx-com  
Namespace: cert-manager  
Labels:   
Annotations:   
API Version: [cert-manager.io/v1](http://cert-manager.io/v1)  
Kind: Certificate  
Metadata:  
Creation Timestamp: 2024-12-02T09:15:55Z  
Generation: 1  
Resource Version: 371697  
UID: 4dda1a78-f6ed-4328-89f6-4af90eb321ee  
Spec:  
Common Name: [test.50gramx.com](http://test.50gramx.com)  
Dns Names:  
[test.50gramx.com](http://test.50gramx.com)  
Issuer Ref:  
Kind: ClusterIssuer  
Name: letsencrypt-staging  
Secret Name: tls-secret  
Status:  
Conditions:  
Last Transition Time: 2024-12-02T09:15:55Z  
Message: Issuing certificate as Secret does not exist  
Observed Generation: 1  
Reason: DoesNotExist  
Status: False  
Type: Ready  
Last Transition Time: 2024-12-02T09:15:55Z  
Message: Issuing certificate as Secret does not exist  
Observed Generation: 1  
Reason: DoesNotExist  
Status: True  
Type: Issuing  
Next Private Key Secret Name: test-50gramx-com-4p4gw  
Events:

---

<div class="post-metadata">

### Author: ![jdpaterson](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/jdpaterson/32/17245_2.png) [@jdpaterson](https://discuss.kubernetes.io/u/jdpaterson)
#### Post date: [May 3, 2025, 3:13pm UTC](https://discuss.kubernetes.io/t/addon-cert-manager/21122/19 "2025-05-03T15:13:23Z")

</div>

There is a typo in the name of the private key, `name: lets-encrypt-priviate-key` which is an eyesore, is there a way to rename that private key without repeating the whole process?
