Issue with pulling image from Docker REgistry

General Discussions
1

Asking for help? Comment out what you need so we can get more information to help you!

Cluster information:

Kubernetes version: Client Version: v1.30.9
Cloud being used: Bare metal
Installation method:
Host OS: RHEL 9
CNI and version: Calico
CRI and version: containerd.io-1.7.25-3.1.el9.x86_64

I recently stand up a small K8s cluster for experiments. It contains 1 master node, 2 worker nodes, and 1 separate Docker registry server. The issue I’m facing is that I am not able to pull the image on the Docker server using CRI. It looks to work but it is not pulling the image down. The total download is 0 Bytes.

===========================================
[root@kubewradls01 ~]# ctr image pull --skip-verify --plain-http kuberradls01.inet.local:5000/ubuntu:latest
WARN[0000] DEPRECATION: The mirrors property of [plugins."io.containerd.grpc.v1.cri".registry] is deprecated since containerd v1.5 and will be removed in containerd v2.1. Use config_path instead.
WARN[0000] DEPRECATION: The configs property of [plugins."io.containerd.grpc.v1.cri".registry] is deprecated since containerd v1.5 and will be removed in containerd v2.1. Use config_path instead.
INFO[0000] host will try HTTPS first since it is configured for HTTP with a TLS configuration, consider changing host to HTTPS or removing unused TLS configuration host=“kuberradls01.inet.local:5000”
INFO[0000] host will try HTTPS first since it is configured for HTTP with a TLS configuration, consider changing host to HTTPS or removing unused TLS configuration host=“kuberradls01.inet.local:5000”
kuberradls01.inet.local:5000/ubuntu:latest: resolved |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:104f82606ea66da00e6cfecbcccdb53ba4238a7057bed809f004107ad8e90c97: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:107a4fb0af3829758fca8bb809bb86ccb87dd630e4d61110103466886f7b0746: done |++++++++++++++++++++++++++++++++++++++|
config-sha256:a04dc4851cbcbb42b54d1f52a41f5f9eca6a5fd03748c3f6eb2cbeb238ca99bd: done |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.1 s total: 0.0 B (0.0 B/s)
unpacking linux/amd64 sha256:104f82606ea66da00e6cfecbcccdb53ba4238a7057bed809f004107ad8e90c97…
done: 5.608806ms

It works fine if I use docker pull

===========================================
[root@kubewradls01 ~]# docker pull kuberradls01.inet.local:5000/ubuntu:latest
latest: Pulling from ubuntu
Digest: sha256:104f82606ea66da00e6cfecbcccdb53ba4238a7057bed809f004107ad8e90c97
Status: Image is up to date for kuberradls01.inet.local:5000/ubuntu:latest
kuberradls01.inet.local:5000/ubuntu:latest

Is there anything I can do to troubleshoot this further? This is the log from Docker REgistry when the pull is initiated via CRI

===========================================
time=“2025-02-12T14:30:57.149765932Z” level=info msg=“response completed” go.version=go1.20.8 http.request.host=“kuberradls01.inet.local:5000” http.request.id=7247d10f-3186-4a00-96d0-fc6dc9104a0a http.request.method=HEAD http.request.remoteaddr=“192.168.6.158:45892” http.request.uri=“/v2/ubuntu/manifests/latest” http.request.useragent=“containerd/1.7.25” http.response.contenttype=“application/vnd.docker.distribution.manifest.v2+json” http.response.duration=1.420145ms http.response.status=200 http.response.written=529
192.168.6.158 - - [12/Feb/2025:14:30:57 +0000] “HEAD /v2/ubuntu/manifests/latest HTTP/1.1” 200 529 “” “containerd/1.7.25”

versus docker pull

===========================================
time=“2025-02-12T14:31:13.469125575Z” level=info msg=“response completed” go.version=go1.20.8 http.request.host=“kuberradls01.inet.local:5000” http.request.id=3c609975-1dc1-4129-b697-2c80c3bd3f53 http.request.method=GET http.request.remoteaddr=“192.168.6.158:45926” http.request.uri=“/v2/” http.request.useragent=“docker/27.5.1 go/go1.22.11 git-commit/4c9b3b0 kernel/5.14.0-427.28.1.el9_4.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/27.5.1 (linux))” http.response.contenttype=“application/json; charset=utf-8” http.response.duration=1.042637ms http.response.status=200 http.response.written=2
192.168.6.158 - - [12/Feb/2025:14:31:13 +0000] “GET /v2/ HTTP/1.1” 200 2 “” “docker/27.5.1 go/go1.22.11 git-commit/4c9b3b0 kernel/5.14.0-427.28.1.el9_4.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/27.5.1 \(linux\))”
time=“2025-02-12T14:31:13.472330172Z” level=info msg=“response completed” go.version=go1.20.8 http.request.host=“kuberradls01.inet.local:5000” http.request.id=dcbe370f-ad7a-4447-80a7-c09b6b0c5747 http.request.method=HEAD http.request.remoteaddr=“192.168.6.158:45934” http.request.uri=“/v2/ubuntu/manifests/latest” http.request.useragent=“docker/27.5.1 go/go1.22.11 git-commit/4c9b3b0 kernel/5.14.0-427.28.1.el9_4.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/27.5.1 (linux))” http.response.contenttype=“application/vnd.docker.distribution.manifest.v2+json” http.response.duration=1.008844ms http.response.status=200 http.response.written=529
192.168.6.158 - - [12/Feb/2025:14:31:13 +0000] “HEAD /v2/ubuntu/manifests/latest HTTP/1.1” 200 529 “” “docker/27.5.1 go/go1.22.11 git-commit/4c9b3b0 kernel/5.14.0-427.28.1.el9_4.x86_64 os/linux arch/amd64 UpstreamClient(Docker-Client/27.5.1 \(linux\))”

1 Like
2

I see there seems to be a warning, have you tried to resolve this warning before pulling the image?

3

Hi the warning doesn’t have an effect on the pull itself. I’m using some old configs that will be deprecated in the upcoming releases but they are still relevant as of now/

4

I have a couple follow up questions and may be able to help. WHen you say, separate docker registry server are you creating that locally? Also, how are you attempting to run the pods or deployments? Typically you will need to create imagepull secrets for a local repository otherwise it will attempt to pull a default image from docker.

Private registry link - Pull an Image from a Private Registry | Kubernetes

I do see that the ‘host’ and port number is different within the cri and the docker commands so that is worth looking at as well

5

Hi,

I have a docker registry server running locally in my environment. My aim is to pull to have my kube cluster pull images from that registry server. I’m not sure how you are seeing the hosts and ports are different since they are pulling from the same host and port 5000