Strict confinement

General Discussions microk8s
1

What is strict confinement?

Strict confinement is a snap confinement level that provides complete isolation, up to a minimal access level to the host resources. Strictly confined snaps can not access files, networks, processes, or any other system resource without requesting specific permission. It is therefore deemed safe. Strict confinement uses security features of the Linux kernel, including AppArmor, seccomp, and namespaces to prevent applications and services from accessing the wider system.

How to grant permissions?

Interfaces are the key. Interfaces have two sides a plug and a slot. Plugs can be thought of as resource access requests while slots are the resource access options available on a host. Each snap’s slot is carefully selected by the creator to map resource requirements while plugs are predefined system wide. An interface needs to be connected to be active so connections are made (either automatically at install time or manually) depending on their function. See the full list of interfaces interfaces here.

What does this mean for MicroK8s?

Strict confinement proves to be a great match for the Kubernetes runtime and hosted workloads. Often times these application workloads interact with the host machines in ways that can be unpredictable and might even be insecure. Strict confinement makes sure that this dynamic environment is isolated and that application workloads are doing what they are expected to be doing. Any CVEs, malicious actors, bugs, etc. will be limited to the sandbox setup by strict confinement.

Note that there are caveats to this level of isolation. Some applications that require elevated permissions to critical system resources which might not be allowed to run.

Notable differences between strict and classic MicroK8s versions

  • Workloads requiring “shared mount” point with the host are blocked because they break the expected isolation. For these workloads the classic snap should be used instead (snap install microk8s --classic). Addons affected by this include, cilium, gpu and multus.

  • The MicroK8s team tests the strictly confined snap extensively against a large set of workloads. However this does not guarantee that every possible workload will run unaffected in a strictly confined setup. Should you find one such workload we would appreciate if you opened a github issue so we can review the case and include proper mitigation in future releases. To handle such cases users can use the --devmode flag when installing the strict snap.

  • Due to limitations in how strictly confined services are handled certain microk8s commands that anyone from the microk8s user groups could execute now require elevated permissions.

2

I think we miss another notable difference in this documentation: hostPath volumes pointing to a directory outside of the strictly confined snap.

In a non-strictly-confined microk8s installation, this volume would mount the /writable/path/outside/snap existing host directory inside the pod/container configured volumeMounts.mountPath:

volumes:
  - name: host-path-data-volume
    hostPath:
      path: /writable/path/outside/snap
      type: Directory

But, in a strictly confined microk8s installation, which - to the best of my knowledge - is bound to /var/snap/microk8s/common/default-storage/ for its storage through the hostpath-storage addon, this volume just won’t mount, and a kube describe pods would display this related error:

Warning  FailedMount  76s (x14 over 13m)   kubelet  MountVolume.SetUp failed for volume "host-path-data-volume" : hostPath type check failed: /writable/path/outside/snap is not a directory

And that makes sense: we don’t expect a strictly confined snap to access just any folder on its host! But that breaks one’s k8s deployment on this host :upside_down_face:

What do you think @kjackal? Should we add a “notable difference item” mentioning that, and explaining how to get through it? Or… did I just miss something?

3

For the record, microk8s strictly confined snap has access to $HOME through one of its connection. But ln is not an option (not seen as a directory by microk8s). Obsiouvly, one could sudo mount --bind their /writable/path/outside/snap to their $HOME but then they have to write a snap to automatically re-mount that on every boot, as /etc/fstab is regenerated on every Ubuntu Core boot :upside_down_face:

So the only option I found out to be working is to actually place the data somewhere in $HOME/some-data and use /home/myusername/some-data as the volumes.hostPath.path value.

4

and instead of ans.

One of the slots in this sentence should be a plug.

guarantee instead of guaranty.

Thanks!

1 Like