PSPs and gaining root access on nodes

General Discussions
1

Cluster information:

Kubernetes version: 1.18.3
Cloud being used: baremetal
Installation method: rancher
Host OS: OracleLinux 7.8
CNI and version: calico-cni:v3.13.4
CRI and version: docker 19.3.11

I am using rancher for cluster management and cluster creation (RKE). I have enabled restricted pod security policies in the cluster. If I try to create a deployment with restricted access the replicaset fails to create the pod due to a psp violation.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: privileged-deploy
  name: privileged-deploy
  namespace: psp-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: privileged-deploy
  template:
    metadata:
      labels:
        app: privileged-deploy
    spec:
      containers:
      - image: docker.io/alpine:3.9
        name: shell
        command: [ "nsenter" ]
        args: ["-t", "1", "-m", "-u", "-i", "-n", "sleep", "14000"]
        stdin: true
        tty: true
        securityContext:
          privileged: true
      hostPID: true
      hostNetwork: true
      hostIPC: true

This fails:

Events:
Type Reason Age From Message

Warning FailedCreate 7m35s (x20 over 46m) replicaset-controller Error creating: pods “privileged-deploy-7689c779c5-” is forbidden: unable to validate against any pod security policy: [spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used spec.securityContext.hostPID: Invalid value: true: Host PID is not allowed to be used spec.securityContext.hostIPC: Invalid value: true: Host IPC is not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]

If I deploy a pod directly with certain specs, I can gain root access on nodes.

apiVersion: v1
kind: Pod
metadata:
  name: root-access
spec:
  hostPID: true
  hostIPC: true
  hostNetwork: true
  containers:
  - name: shell
    image: docker.io/alpine:3.9
    command: [ "nsenter" ]
    args: ["-t", "1", "-m", "-u", "-i", "-n", "sleep", "14000"]
    securityContext:
      privileged: true

This creates a pod without any issue. It’s like the psp does not apply to pod spec directly. I can only assume that I am not doing something incorrectly, or this is a bug. It boggles my mind that the deployment fails, but creating a pod directly fails.

2

Can you please check your pod security policy for namespace: default and the default service account? I can’t reproduce this in my env. My setup is onprem k8s

Error from server (Forbidden): error when creating "testpod": pods "root-access" is forbidden: unable to validate against any pod security policy: [spec.securityContext.hostIPC: Invalid value: true: Host IPC is not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed spec.securityContext.hostNetwork: Invalid value: true: Host network is not allowed to be used spec.securityContext.hostPID: Invalid value: true: Host PID is not allowed to be used spec.securityContext.hostIPC: Invalid value: true: Host IPC is not allowed to be used spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed
3

I don’t fully understand your question. Are you asking about a psp in the namespace, or a psp for the namespace called default? The question is a little ambiguous. I am also testing this in a different namespace than default.

Also all of the psps are defined at the cluster level and not in a namespace.