Configuration management in Kubernetes

General Discussions
1

Hello all,

I am in the process of learning Kubernetes and trying to set up my own pod. After doing lots of reading, it seems that the generally accepted best practice for managing configuration for my containers is through the use of configuration maps. Is this true?

I am having a hard time understanding the benefits of configuration maps. Maybe I am missing something. When I try to create a configuration map from a file, it basically creates the object and the object is static; that is, if I go and change the configuration file on disk, it does not change the configuration map. I can then map the files into my containers. The problem is, what if I need to update the configuration and then restart the service/pod? For example in my pod I have squid running with an icap service connected to it. If I need to update the whitelist for example, what I would do in a normal setup is update the file, and then restart the squid service. With a configuration map it seems like I would have to create a new configuration map, update the pod to point to that one, then restart the pod.

In the documentation and tutorial for configuration maps, I didn’t see this scenario covered.

https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#understanding-configmaps-and-pods

Furthermore, the ConfigMaps documentation does not even give a real world example, just a seemingly made up one (game-config). Most of the other Kubernetes documentation/tutorials use the nginx example, which is much more useful and helps in understanding the concepts since you can apply them directly.

Can someone help me understand? I am tempted to just use another route (like shared volumes) but I wanted to make sure I am understanding the problem before I depart from a best practice.

2

Hi Justin.

Config Maps are the standard way to manage configuration of applications in K8s. It supports declarative approach of describing the environment. In my opinion they are very handy - can be mounted as files, can be exposed as env variables, can be shared between applications, managing is almost identical with managing secrets etc.

Some hints for you:

  1. ConfigMaps mounted as files are synced by K8s periodically (Configure a Pod to Use a ConfigMap - Kubernetes).
    So the application can detect changes and reloads itself or you can attach a sidecar (an additional container within the same pod) which reloads service (eg. your squid service) if it happens.
    For example - Prometheus Operator starts Pod with Prometheus and with sidecards to handle the problem: https://github.com/coreos/prometheus-operator/tree/master/cmd/prometheus-config-reloader

  2. Kustomize (embedded in the latest versions of kubectl) has something named ConfigGenerator which generates the new ConfigMap and updates refs in Pods when values are changed.
    More information: https://github.com/kubernetes-sigs/kustomize/blob/master/examples/configGeneration.md
    It’s good enough if the configuration is not changed very often.

Best,
Tomasz Prus

3

Tomasz, do you mean once the configuration map is changed manually, or when the files that the configuration map is created from are changed on the disk? Because it seems to be that it would be useful to be able to store the actual configuration files used by the application as code.

4

It may work in this way but be it doesn’t out of the box. You have to configure CI/CD pipelines or something like GitOps to sync your repository with K8s documents and then k8s will update your configuration files inside container if the configmap changes.