Understanding CSI architecture and communication

General Discussions
2

Hi candlerb,

(Aside: why use a hostPath volume for the socket, rather than putting all these components in a single pod which communicate via an emptyDir? This isn’t really important though)

Sorry, I don’t know this reason.

My main question is this. How would you go about changing this so that it could provision volumes on multiple nodes?

I don’t think this is possible.
GitHub - kubernetes-csi/csi-driver-host-path: A sample (non-production) CSI Driver that creates a local directory as a volume on a single node works as below:

I mean GitHub - kubernetes-csi/csi-driver-host-path: A sample (non-production) CSI Driver that creates a local directory as a volume on a single node is not intended to operate on multiple nodes.

I recommend reading other CSI drivers(e.g. GitHub - kubernetes-sigs/aws-ebs-csi-driver: CSI driver for Amazon EBS https://aws.amazon.com/ebs/).

And you should read Kubernetes CSI’s design proposal.
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md

CSI plugin works as below:

960×540 62.5 KB

  • The application of Daemonset Pod of the image works on all nodes.
  • The application of StatefulSet/Deployment Pod of the image works as Kubernetes controller.
  • In the DaemonSet Pod, CSI Driver communicates from Kubelet using gRPC and a socket.
  • The node-driver-registrar registers a CSI Driver to Kubelet as a plugin.
  • In the StatefulSet/Deployment Pod, CSI Driver communicates from CSI sidecar applications using gRPC and a socket.
  • CSI sidecar applications work as a Kubernetes controller.
  • For example, if create a PVC resource, external-provisoner watch this event, and call CreateVolume rpc to CSI Driver.

So

Should the hostpath-plugin on each node expose its grpc endpoint as a “service”?

No, because CSI driver communicates using gRPC and a socket.

If so, is it responsible for securing/authenticating connections over that service?

No.

Can RBAC be used to lock down access to these services?

No, but you need setting RBAC for CSI sidecar applications to CSI sidecar applications can communicate to api-server.
(e.g. external-provisioner: https://github.com/kubernetes-csi/external-provisioner/blob/6019c43382549945cf7b80f1bffa826c6d14392c/deploy/kubernetes/rbac.yaml)

1 Like