I have a use-case where a K8s controller watches PVs, periodically runs some logic/function on underlying PV’s data and stores logic/function result in K8s CRD.
I came to a design where a controller which watches PVs spawns an aux Pod to which it attaches PV (via PVC) and runs logic/function in it. I have following requirement for the controller and aux Pod:
- logic/function in the aux Pod may require extra input parameters
- logic/function in the aux Pod may generate a result, like error code and description in case of failure or in case of success it may return a reference to an external resource which was created as a result of logic/function execution.
- logic/function in the aux Pod is not K8s aware, so it cannot read or write from/to K8s CRDs directly.
- Controller <-> aux Pod communication latency is not a concern
Given the design and requirements I have 2 questions:
- What’s the best way to pass extra input parameters to this aux Pod function?
I suspect env variables might be the right mechanism for this, but want to double-check.
- What’s the best way to communicate logic/function result back to the controller?
One of the options is to use network communication (like REST), but I’m afraid securing network communication properly including auth* might be tricky and I’d like to avoid that. Another option I considered is to use remote pod/container copy feature (like kubectl cp) which looks a bit hacky but is not hard to implement.
WDYT? I’d appreciate your thoughts and ideas on the design.