Security context `fsUser` to not loose write access on mounted filesystem

General Discussions
, ,
1

Cluster information:

Kubernetes version:
Client Version: version.Info{Major:“1”, Minor:“21”, GitVersion:“v1.21.1”, GitCommit:“5e58841cce77d4bc13713ad2b91fa0d961e69192”, GitTreeState:“clean”, BuildDate:“2021-05-12T14:11:29Z”, GoVersion:“go1.16.3”, Compiler:“gc”, Platform:“darwin/amd64”}
Server Version: version.Info{Major:“1”, Minor:“20”, GitVersion:“v1.20.7”, GitCommit:“132a687512d7fb058d0f5890f07d4121b3f0a2e2”, GitTreeState:“clean”, BuildDate:“2021-05-12T12:32:49Z”, GoVersion:“go1.15.12”, Compiler:“gc”, Platform:“linux/amd64”}
Cloud being used: (put bare-metal if not on a public cloud)
Installation method: minikube
Host OS: macOS

Hi everybody,
I have a long time problem in my development environment, that I cannot wrap my head around on.

When I mount a volume from my host onto the app container (hostPath), in order to share the source code (without having to rebuild via Dockerfile and re-deploy on the cluster), I always encounter some permissions issue.

As far as you know, Is it possible to:

  • run the image and commands as non-root user app (high UID: 50000000; high GID: 50000000)
  • keep application source code with minimum permissions; only the owner can write (/app drwxr-xr-x app app)
  • mount a volume from the host, with hostPath, over /app in the container; so during development I have write access from outside the container.
    • NOTE: Docker will set the whole mounted path owner & group to UID/GID 1000:1000
  • have commands executed (like kubectl exec deploy/myapp -- touch foobar) to not fail with permissions denied, because of the running user app having UID different from the one set with the mounted volume.

I don’t want to relax the file permissions with chmod g+w because writing would fail anyway with newly created files:
either the FROM image in the Dockerfile and my host OS use umask 022; so each new file will grant write access only to the owner, not for the group (otherwise, fsGroup would solve my issue).

I tried to play with securityContext.fsGroup in order to set GID 50000000 but what I would actually need to retain write permissions should be something like securityContext.fsUser.
Does Kubernetes has anything like that? If not, why?

How can the owner retain write permissions, when mounting via hostPath?

2

Broke this up into sections. I think I covered the bulk of your questions.

User ID and Group ID

I don’t really know what the min and max ID values are, but I do know UID and GUIDs are limited to datatypes being used. POSIX compliant OSes are expected to use at least unsigned integers. As for Kubernetes, It looks like int64 to me.

To set the UID and GID, look into setting values in the securityContext for your containers.

From what I understand, this wont magically set permissions for your files for a mounted volume. You may need to do chown/chmod operations with an initContainer in advance in some cases. In your particular use-case, it seems fine.

Mounting /app

Yeah hostPath just does that. File permissions shouldn’t change?

kubectl exec

This should be a non-problem if your container is setup to run as a specific UID and GID.

Developing stuff in Kubernetes

There are tools that help you do development. Mounting /app just to develop an app isn’t very portable if you have to work with other people. People will want to use different deployments locally like Docker for Desktop or Minikube.

In my experience, I have really liked Skaffold for container development. It’s maintained by Google and it builds and redeploys your development container as you make changes.

For app development though, rebuilding and reloading is very slow. I like devspace by Loft, for app development. They’re putting a lot of really useful open source tools out there for Kubernetes. DevSpace can actively sync files to a development container OR just rebuild the container OR do a hybrid of both things depending on which files you modify.