"Pod as a persistent volume data based function" design

General Discussions
1

Hey all,

I have a use-case where a K8s controller watches PVs, periodically runs some logic/function on underlying PV’s data and stores logic/function result in K8s CRD.

I came to a design where a controller which watches PVs spawns an aux Pod to which it attaches PV (via PVC) and runs logic/function in it. I have following requirement for the controller and aux Pod:

  1. logic/function in the aux Pod may require extra input parameters
  2. logic/function in the aux Pod may generate a result, like error code and description in case of failure or in case of success it may return a reference to an external resource which was created as a result of logic/function execution.
  3. logic/function in the aux Pod is not K8s aware, so it cannot read or write from/to K8s CRDs directly.
  4. Controller <-> aux Pod communication latency is not a concern

Given the design and requirements I have 2 questions:

  1. What’s the best way to pass extra input parameters to this aux Pod function?
    I suspect env variables might be the right mechanism for this, but want to double-check.
  2. What’s the best way to communicate logic/function result back to the controller?
    One of the options is to use network communication (like REST), but I’m afraid securing network communication properly including auth* might be tricky and I’d like to avoid that. Another option I considered is to use remote pod/container copy feature (like kubectl cp) which looks a bit hacky but is not hard to implement.

WDYT? I’d appreciate your thoughts and ideas on the design.

Thank you.