Registry images

Kubernetes manages containerised applications based on images. These images can be created locally, or more commonly are fetched from a remote image registry. The following documentation explains how to use MicroK8s with local images, or images fetched from public or private registries.

A familiarity with building, pushing and tagging container images will be helpful. These examples use Docker but you can use your preferred container tool chain.

To install Docker on Ubuntu 18.04:

sudo apt-get install docker.io

Add the user to the docker group:

sudo usermod -aG docker ${USER}

Open a new shell for the user, with updated group membership:

su - ${USER}

The Dockerfile we will be using is:

FROM nginx

To build the image tagged with mynginx:local, navigate to the directory where Dockerfile is and run:

docker build . -t mynginx:local

This will generate a new local image tagged mynginx:local.

Working with locally built images without a registry

When an image is built it is cached on the Docker daemon used during the build. Having run the docker build . -t mynginx:local command, you can see the newly built image by running:

docker images

This will list the images currently known to Docker, for example:

REPOSITORY          TAG                 IMAGE ID            SIZE
mynginx             local               1fe3d8f47868        16.1MB

The image we created is known to Docker. However, Kubernetes is not aware of the newly built image. This is because your local Docker daemon is not part of the MicroK8s Kubernetes cluster. We can export the built image from the local Docker daemon and “inject” it into the MicroK8s image cache like this:

docker save mynginx > myimage.tar
microk8s ctr image import myimage.tar

Note that when we import the image to MicroK8s we do so under the k8s.io namespace.

Now we can list the images present in MicroK8s:

microk8s ctr images ls

At this point we are ready to microk8s kubectl apply -f a deployment with this image:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: mynginx:local
        imagePullPolicy: Never
        ports:
        - containerPort: 80

We reference the image with image: mynginx:local. Kubernetes will behave as though there is an image in docker.io (the Dockerhub registry) for which it already has a cached copy. This process can be repeated any time changes are made to the image. Note that containerd will not cache images with the latest tag so make sure you avoid it.

ImagePullPolicy: Never doesn’t seem to work on the current version. This document https://itnext.io/working-with-image-registries-and-containerd-in-kubernetes-63c311b86368 more or less mirrors the documentation but omits that.

Thanks for the feedback @serversonic
That does seem to be the correct policy for using local images, but I’ll check into it

What do you mean by that? Does the pod crash or is it failing silently? Also, by current version you mean Kubernetes v1.18.2? Thank you.

microk8s version is 1.18.2, installed as per the docs.
I followed this page: https://microk8s.io/docs/registry-images

I can see from microk8s ctr images ls that I have an image as follows:
ee6992c2ed0f4628fcef75751048927bcd6b1cee89c79f6acb63ca5474d5a 11.2 MiB linux/amd64,linux/arm,linux/arm64,linux/ppc64le,linux/s390x io.cri-containerd.image=managed docker.io/library/mynginx:local application/vnd.oci.image.manifest.v1+json sha256:49b

But creating a deployment using the YAML given results in this:

kubectl get pods nginx-deployment-6b96d9cb5d-ldg5k 0/1 ErrImageNeverPull 0 54m

I can see the image is being written to syslog here:
Jun 9 10:18:37 cluster01 microk8s.daemon-containerd[4486]: time="2020-06-09T10:18:37.824339360Z" level=info msg="ImageUpdate event &ImageUpdate{Name:docker.io/library/mynginx:local,Labels:map[string]string{io.cri-containerd.image: managed,},}"

While I’ve been writing this I deleted the pod that had failed, and then the deployment, then tried it again and now it’s worked. The error I was getting in syslog was along the lines of the image not available in the local registry and policy meaning it wouldn’t be pulled externally.

I’m looking at this as a experienced sysadmin who is new to kubernetes
and starting from scratch, and it seems like a troubleshooting issue which would be (well, is) frustrating if the docs are a tutorial but on the other hand I can understand that microk8s is meant to be a ‘just works’ version of kubernetes and I’m fine with that too, so it feels like a documentation issue rather than a bug but I’m not really sure what a viable resolution would be.

Thanks again for mentioning this - glad you resolved your issue.
I’m going to run through this page and see what useful extra detail and troubleshooting we can add

It’s a bit strange that this article does not go any further. It stops at deploying the image, but does not explain how to verify that the container is actually running, and how to reach the Nginx service that was deployed…

Via a separate tutorial I installed the dashboard, but otherwise I wouldn’t be able to figure out how to actually access/use the deployed service.

BTW, I’m new to k8s and MicroK8s. Earlier I tried deploying a local image using Minikube, but got stuck trying to use a local container repository (which itself needs to run inside Minikube). So I love how much easier this is with MicroK8s!