AKS + on prem registry + self signed cert

CrankyCoder · January 15, 2020, 12:59am

I have no idea if AKS questions can even be asked on here. If not, my apologies.

Scenario:

I have aks setup. 3 nodes. I have a private docker registry running in artifactory on prem. There is a vpn tunnel between azure and on prem.

I created a deployment to pull an image out of our private registry. Here is where the fun comes in. Corp Policy dictates that internal servers use certificates from our enterprise CA. So the cert on the artifactory box is trusted by default by internal machines and any machine i can get the CA chain added as a trust… But the AKS nodes. Is there any way to provide the aks nodes a custom docker daemon config to see this as an insecure registry? or a way to add our CA to the nodes?

So far i am not seeing any way to do anything to the aks nodes short of getting off aks and just doing kubernetes straight on VMs in azure.

Thanks!

macintoshprime · January 15, 2020, 4:30pm

I’ve been using this pattern to load the CA into the Azure Host nodes for our on-prem Harbor Registry and it’s been working great

github.com

coreos/tectonic-docs/blob/master/Documentation/admin/add-registry-cert.md

# Add Registry Certificate to Tectonic Worker Nodes

Many enterprises use self-signed certificates to protect container registries. On kubernetes this can present a problem as docker requires that self-signed certificates be nested under the `/etc/docker/certs.d` on every host that will pull from a registry with a self-signed certificate.

This restriction can be solved with a DaemonSet that copies the `ca.crt` file to the needed directory on each host.

A secret will be mounted into the DaemonSet as a file and then copied. This secret must include the base64 encoded contents of the root CA (pem format) used to sign the container registry certs.

```bash
$ base64 -w 0 rootCA.pem
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURWekNDQWorZ0F3SUJBZ0lKQUxRd3FGRWVpakdyTUEwR0NTcUdTSWIzRFFFQkN3VUFNRUl4Q3pBSkJnTlYKQkFZVEFsaFlNUlV3[...]
```


#### registry-secret.yaml

```yaml
apiVersion: v1
kind: Secret
metadata:

This file has been truncated. show original

CrankyCoder · January 15, 2020, 5:08pm

I think i have seen something similar to this before. This basically just creates a pod on every node that copies the cert info from a secret to a file on the server correct?

At this point, I firmly believe i am going to have to try it lol.

Glad to see atleast someone else has run in to this and came up with something that works.

Ill report back!

macintoshprime · January 15, 2020, 5:36pm

It seems a little hacky but it was azure support who gave me that recommendation

Hope it works!

CrankyCoder · January 15, 2020, 5:48pm

oh now that’s just fantastic LOL

if it works. Ill take it!

CrankyCoder · January 18, 2020, 12:55am

You can add worked with AKS to the list!!! lol thanks a ton for that!

parwinderjit_singh · July 17, 2020, 2:46pm

Hi Guys,

I am creating new AKS cluster and trying to pull images from on-prem artifactory. In the above method where the busybox image is coming from ? Is it from the artifactory or from ACR ?

macintoshprime · July 18, 2020, 3:26pm

The busybox container would be pulling from here.

Topic		Replies	Views
Configuring insecure registry for k8s microk8s	4	12423	January 28, 2019
Create and use a private insecure registry inside k8s cluster General Discussions	2	2799	July 10, 2020
Using an intermediate CA whose certificate is signed by a self-signed root CA certificate General Discussions	0	870	May 3, 2022
Microk8s can't pull image from a private registry with ssl self signed certificate microk8s docs	4	8171	April 22, 2022
How to add self signed cert to pod or container ca root General Discussions	0	5785	April 16, 2021

AKS + on prem registry + self signed cert

Related topics