# How to use the built-in registry

**URL:** <https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274>\
**Category:** microk8s\
**Tags:** docs\
**Created:** [June 4, 2020, 3:25pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274 "2020-06-04T15:25:17Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\
**Post date:** [June 4, 2020, 3:25pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/1 "2020-06-04T15:25:17Z")

</div>

Having a private Docker registry can significantly improve your productivity by reducing the time spent in uploading and downloading Docker images. The registry shipped with MicroK8s is hosted within the Kubernetes cluster and is exposed as a NodePort service on port `32000` of the `localhost`. Note that this is an insecure registry and you may need to take extra steps to limit access to it.

# Working with MicroK8s’ built-in registry

You can install the registry with:

```bash
microk8s enable registry

```

The add-on registry is backed up by a `20Gi` persistent volume is claimed for storing images. To satisfy this claim the storage add-on is also enabled along with the registry.

From version 1.18.3 it is also possible to specify the amount of storage to be added. E.g., to use 40Gi:

```auto
microk8s enable registry:size=40Gi

```

The containerd daemon used by MicroK8s is configured to trust this insecure registry. To upload images we have to tag them with `localhost:32000/your-image` before pushing them:

We can either add proper tagging during build:

ⓘ **Note:** The `:registry` tag used below is just an example. Any tag can be used. However, containerd will cache them when pulling from the registry, and subsequent pushes won’t have any effect on Pods running inside of MicroK8s. You can either manually update the containerd image with `microk8s ctr image pull localhost:32000/mynginx:registry`, or use the `:latest` (or no) tag, which containerd will not cache.

```bash
docker build . -t localhost:32000/mynginx:registry

```

Or tag an already existing image using the image ID. Obtain the ID by running:

```bash
docker images

```

The ID is listed in the output:

```no-highlight
REPOSITORY TAG IMAGE ID SIZE
mynginx local 1fe3d8f47868 16.1MB
....

```

Then use the `tag` command:

```bash
docker tag 1fe3d8f47868 localhost:32000/mynginx:registry

```

Now that the image is tagged correctly, it can be pushed to the registry:

```bash
docker push localhost:32000/mynginx

```

Pushing to this insecure registry may fail in some versions of Docker unless the daemon is explicitly configured to trust this registry. To address this we need to edit `/etc/docker/daemon.json` and add:

```json
{
  "insecure-registries" : ["localhost:32000"]
}

```

The new configuration should be loaded with a Docker daemon restart:

```bash
sudo systemctl restart docker

```

At this point we are ready to `microk8s kubectl apply -f` a deployment with our image:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: localhost:32000/mynginx:registry
        ports:
        - containerPort: 80

```

## What if MicroK8s runs inside a VM?

Often MicroK8s is placed in a VM while the development process takes place on the host machine. In this setup pushing container images to the in-VM registry requires some extra configuration.

Let’s assume the IP of the VM running MicroK8s is `10.141.241.175`. When we are on the host the Docker registry is not on `localhost:32000` but on `10.141.241.175:32000`. As a result the first thing we need to do is to tag the image we are building on the host with the right registry endpoint:

```bash
docker build . -t 10.141.241.175:32000/mynginx:registry

```

If we immediately try to push the `mynginx` image we will fail because the local Docker does not trust the in-VM registry. Here is what happens if we try a push:

```bash
docker push 10.141.241.175:32000/mynginx

```

```no-highlight
The push refers to repository [10.141.241.175:32000/mynginx]
Get https://10.141.241.175:32000/v2/: http: server gave HTTP response to HTTPS client

```

We need to be explicit and configure the Docker daemon running on the host to  
trust the in-VM insecure registry. Add the registry endpoint in  
`/etc/docker/daemon.json`:

```json
{
  "insecure-registries" : ["10.141.241.175:32000"]
}

```

Then restart the docker daemon on the host to load the new configuration:

```bash
sudo systemctl restart docker

```

We can now `docker push 10.141.241.175:32000/mynginx` and see the image getting uploaded. During the push our Docker client instructs the in-host Docker daemon to upload the newly built image to the `10.141.241.175:32000` endpoint as marked by the tag on the image. The Docker daemon sees (on `/etc/docker/daemon.json`) that it trusts the registry and proceeds with uploading the image.

Consuming the image from inside the VM involves no changes:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: localhost:32000/mynginx:registry
        ports:
        - containerPort: 80

```

Reference the image with `localhost:32000/mynginx:registry` since the registry runs inside the VM so it is on `localhost:32000`.

## Using the local registry from another node in a MicroK8s cluster

If you have joined up other machines into a cluster with the machine that has the registry, you need to change the configuration files to point to the IP of the master node:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: <IP of the master node>:32000/mynginx:registry
        ports:
        - containerPort: 80

```

ⓘ **Note:** You will also need to manually edit the containerd TOML on each of the worker nodes to point to and trust this custom registry. Instructions for this are in the [private registry instructions](https://discuss.kubernetes.io/t/working-with-a-private-registry/11273) in the **Configuring Micro8s** section.

---

<div class="post-metadata">

**Author:** ![clicky](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/clicky/32/5613_2.png) [@clicky](https://discuss.kubernetes.io/u/clicky)\
**Post date:** [August 3, 2020, 6:25am UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/2 "2020-08-03T06:25:02Z")

</div>

> [@evilnick](#):
>
> The registry shipped with MicroK8s is hosted within the Kubernetes cluster and is exposed as a NodePort service on port `32000` of the `localhost`.

So, in:

> [@evilnick](#):
>
> Using the local registry from another node in a MicroK8s cluster

> [@evilnick](#):
>
> If you have joined up other machines into a cluster with the machine that has the registry, you need to change the configuration files to point to the IP of the master node

Is this still valid to say you need to put master node’s IP as there’s a repository service with nodePort at 32000?

---

<div class="post-metadata">

**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\
**Post date:** [August 3, 2020, 12:22pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/3 "2020-08-03T12:22:08Z")

</div>

I believe so, as the actual contents of the registry are only available at the master node. @kjackal?

---

<div class="post-metadata">

**Author:** ![clicky](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/clicky/32/5613_2.png) [@clicky](https://discuss.kubernetes.io/u/clicky)\
**Post date:** [August 3, 2020, 1:06pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/4 "2020-08-03T13:06:29Z")

</div>

I thought that actual content is where pod is deployed (it is not necessarily master node!). Documentation is not necessarily wrong as you can still use master node’s IP/domain name, but it doesn’t seem needed as because of NodePort every node can access the registry at localhost at port 32000.

BTW I did test it just for fun…

---

<div class="post-metadata">

**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\
**Post date:** [August 4, 2020, 2:37pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/5 "2020-08-04T14:37:26Z")

</div>

cool, I will take your word for it, but I will wait for @kjackal to confirm how its supposed to work 😉

---

<div class="post-metadata">

**Author:** ![kjackal](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/kjackal/32/1750_2.png) [@kjackal](https://discuss.kubernetes.io/u/kjackal)\
**Post date:** [August 10, 2020, 9:24am UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/6 "2020-08-10T09:24:42Z")

</div>

@clicky is right that the registry will be available on all nodes on localhost.

Note that setting up the build in registry in its current state in a cluster is not something we recommend because the storage used to store images is local so if the node hosting the registry pod is removed and/or the registry pod gets recreated on a different node all images will be lost and the registry will have to be repopulated.

It would also make sense to put the registry behind a floating IP via a load balancer (see the metallb addon)

---

<div class="post-metadata">

**Author:** ![clicky](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/clicky/32/5613_2.png) [@clicky](https://discuss.kubernetes.io/u/clicky)\
**Post date:** [August 10, 2020, 10:01am UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/7 "2020-08-10T10:01:14Z")

</div>

Thanks! I just wanted to point out that documentation is, maybe, slightly behind the code (as isn’t it always! 🙂 )

I’ve easily sorted that problem out by adding persistent volume

```
apiVersion: v1
kind: PersistentVolume
metadata:
  name: registry-nfs
spec:
  capacity:
    storage: 50Gi
  accessModes:
    - ReadWriteMany
  storageClassName: nfs
  nfs:
    server: 192.168.4.10
    path: "/srv/k8s/volumes/registry"

```

and pointed to it in `PersistentVolumeClaim` of `container-registry.registry-claim`…

---

<div class="post-metadata">

**Author:** ![a0s](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/a0s/32/6212_2.png) [@a0s](https://discuss.kubernetes.io/u/a0s)\
**Post date:** [November 9, 2020, 7:31pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/8 "2020-11-09T19:31:09Z")

</div>

There is some security problem with using NodePort to access the “insecure registry” outside. The NodePort is binding to `0.0.0.0:32000`, not `localhost:32000`. There is no way to limit NodePort to localhost only:

> <https://github.com/kubernetes/kubernetes/issues/21070>
>
> For now, we are listening on any interface for nodePorts.
> It can be a problem for machines with a public and private...

  

> <https://github.com/kubernetes/kubernetes/issues/90131>
>
> I'd like to limit the address of nodePorts of some services.
> Thanks to --bind-address (#21070), I can serve the nodePorts of all...

Unfortunately, the solution with “docker image save/import tar” is veeeeery slow.

In case you still want to build images on the same hosts where microk8s is running you can use `kubectl port-forward` to access the registry. You can even run `kubectl port-forward` under `systemd` and able to access to the registry at any time. All you need to do is get the original [registry.yaml](https://github.com/ubuntu/microk8s/blob/6eeb85c6c67281c12df7b07b61e4889c2f9d2b52/microk8s-resources/actions/registry.yaml), change NodePort to regular ClusterIP:

```auto
apiVersion: v1
kind: Service
metadata:
  labels:
    app: registry
  name: registry
  namespace: container-registry
spec:
  type: ClusterIP
  selector:
    app: registry
  ports:
    - name: registry
      port: 5000

```

Then apply it and use

```auto
microk8s kubectl \
  --namespace=container-registry \
  port-forward service/registry 5000:5000

```

---

<div class="post-metadata">

**Author:** ![gkapagunta](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/gkapagunta/32/8058_2.png) [@gkapagunta](https://discuss.kubernetes.io/u/gkapagunta)\
**Post date:** [July 1, 2021, 8:14pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/9 "2021-07-01T20:14:42Z")

</div>

> I’ve easily sorted that problem out by adding persistent volume

@clicky Did you have to modify the registry.yml file ([https://github.com/ubuntu/microk8s/blob/master/microk8s-resources/actions/registry.yaml](https://github.com/ubuntu/microk8s/blob/master/microk8s-resources/actions/registry.yaml)) and replace “claimName: registry-claim” with “claimName: registry-nfs” on line #57? Also, did you face any issues with namespaces ( The registry seems to be deployed to a separate namespace “container-registry”)

---

<div class="post-metadata">

**Author:** ![ahasenack](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/ahasenack/32/8600_2.png) [@ahasenack](https://discuss.kubernetes.io/u/ahasenack)\
**Post date:** [September 19, 2021, 8:02pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/10 "2021-09-19T20:02:54Z")

</div>

Hi all,

My setup is microk8s v1.21.3 in a cluster of 3 VMs.

I believe I followed the instructions correctly, and I was able to push my image to the microk8s registry using `docker push` after using the `insecure-registries` config in the host.

My problem now is that deploying a pod using that image is getting the same “http response to https client” error that I got on the host before the `daemon.json` change for `insecure-registries`:

```auto
  Warning Failed 5m22s (x4 over 6m52s) kubelet Failed to pull image "g-k8s:32000/lds:devel": rpc error: code = Unknown desc = failed to pull and unpack image "g-k8s:32000/lds:devel": failed to resolve reference "g-k8s:32000/lds:devel": failed to do request: Head "https://g-k8s:32000/v2/lds/manifests/devel": http: server gave HTTP response to HTTPS client

```

The runtime on the node is containerd (as installed by microk8s), so I believe I now have to do the same kind of “insecure-registries” configuration that I did for docker on the host, but for containerd inside the VM. The microk8s doc doesn’t seem to cover that, or I missed it. Any tips?

---

<div class="post-metadata">

**Author:** ![ahasenack](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/ahasenack/32/8600_2.png) [@ahasenack](https://discuss.kubernetes.io/u/ahasenack)\
**Post date:** [September 19, 2021, 8:09pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/11 "2021-09-19T20:09:44Z")

</div>

On the microk8s node, I can pull that image manually if I pass `--plain-http` to the command:

```auto
root@g-k8s:~# microk8s ctr images pull g-k8s:32000/lds:devel
ctr: failed to resolve reference "g-k8s:32000/lds:devel": failed to do request: Head "https://g-k8s:32000/v2/lds/manifests/devel": http: server gave HTTP response to HTTPS client
root@g-k8s:~# microk8s ctr images pull --plain-http g-k8s:32000/lds:devel
g-k8s:32000/lds:devel: resolved |++++++++++++++++++++++++++++++++++++++| 
manifest-sha256:41337115fca870226f1a5fbfcaeac4b1206c9821fc84f272e44c086b42f50693: done |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:9578fb9701e5b67dff4c21b498d0bfd220a38432f51006d3b8ae70fffe97ae79: done |++++++++++++++++++++++++++++++++++++++| 
config-sha256:7d004e8bea6eb486eb91b37f886dfa0fc85b250fba753e07addf4917973ebf32: done |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:f22ccc0b8772d8e1bcb40f137b373686bc27427a70c0e41dd22b38016e09e7e0: exists |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:3cf8fb62ba5ffb221a2edb2208741346eb4d2d99a174138e4afbb69ce1fd9966: exists |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:e80c964ece6a3edf0db1cfc72ae0e6f0699fb776bbfcc92b708fbb945b0b9547: exists |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:c67f49d56deb603bea8eb139eb81116c36562a5e7cbea8ee342731345e7c1388: done |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:5cdd5069fdb7f0aafb1571b49f7ec9338244f86fc142d745406efa751cfdb1eb: done |++++++++++++++++++++++++++++++++++++++| 
layer-sha256:e5a7d274bee24b995d6831261e882ad617bc21ff60e907ab710f58162962488f: done |++++++++++++++++++++++++++++++++++++++| 
elapsed: 7.7 s total: 441.6 (57.3 MiB/s)    
...

```

---

<div class="post-metadata">

**Author:** ![ahasenack](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/ahasenack/32/8600_2.png) [@ahasenack](https://discuss.kubernetes.io/u/ahasenack)\
**Post date:** [September 19, 2021, 8:22pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/12 "2021-09-19T20:22:09Z")

</div>

Ok, I checked `/var/snap/microk8s/current/args/containerd-template.toml` and it has an entry for `localhost:32000`, but not `g-k8s:32000`. I guess the last section of [https://microk8s.io/docs/registry-built-in](https://microk8s.io/docs/registry-built-in) needs to also explain that one has to change that url in each node from `localhost` to the actual node ip, or else it won’t match the image url used to push.

**UPDATE (this forum won’t let me add another reply, how rude)**

Oh, ok, I’m sorry, the doc does say that in its last line:

> And you need to manually edit the containerd TOML on the worker machines, per [the private registry instructions](https://microk8s.io/docs/registry-private) to trust the insecure registry.

I missed that, since all other instructions had nice yaml examples up to this point.

---

<div class="post-metadata">

**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\
**Post date:** [September 22, 2021, 2:45pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/13 "2021-09-22T14:45:30Z")

</div>

thanks. I’ll try and make that stand out better

---

<div class="post-metadata">

**Author:** ![khteh](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/khteh/32/2106_2.png) [@khteh](https://discuss.kubernetes.io/u/khteh)\
**Post date:** [September 26, 2021, 6:17am UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/14 "2021-09-26T06:17:52Z")

</div>

Is it possible to change the size of the PV used later on after registry addon has been added?

---

<div class="post-metadata">

**Author:** ![Finlay\_Weber](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/finlay_weber/32/12840_2.png) [@Finlay\_Weber](https://discuss.kubernetes.io/u/Finlay_Weber)\
**Post date:** [July 7, 2023, 5:19am UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/15 "2023-07-07T05:19:34Z")

</div>

> [@evilnick](#):
>
> Often MicroK8s is placed in a VM while the development process takes place on the host machine. In this setup pushing container images to the in-VM registry requires some extra configuration.
> 
> Let’s assume the IP of the VM running MicroK8s is `10.141.241.175`. When we are on the host the Docker registry is not on `localhost:32000` but on `10.141.241.175:32000`. As a result the first thing we need to do is to tag the image we are building on the host with the right registry endpoint:

How do I get the IP of the VM running MicroK8s? I am on MacOs

---

<div class="post-metadata">

**Author:** ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)\
**Post date:** [July 7, 2023, 3:57pm UTC](https://discuss.kubernetes.io/t/how-to-use-the-built-in-registry/11274/16 "2023-07-07T15:57:26Z")

</div>

Hi. MicroK8s uses multipass to create VMs on MacOS, so

```bash
multipass list

```

should give you the name of the created VM and

```bash
multipass info <name>

```

will give you info on the instance called `<name>`, including the IP address

Hope that helps, let me know if not