Kubernetes rbac users audit events logging

General Discussions
1

Hi,

I’m trying to find some way to log audit events for each user, i.e whatever the kubectl commands the user executed and the commands which users ran by exec into any of the pods in the cluster?
Please suggest if there is a way to achieve this?

Cluster information:

Kubernetes version: 1.20.x
Cloud being used: AKS
Installation method: Managed
Host OS: Ubuntu
CNI and version: Azure CNI
CRI and version:

2

Have you read the auditing documentation?

3

yes, @protosam. I had gone through this but I don’t think it will log the commands which we run by exec into the pods.

4

The commands interact against the API and that logs API interactions. Logging kubectl isn’t the right solution. What if the command ran is kubectl apply -f -? Someone piped a manifest to be applied and you will have no clue what it was.

5

You’re right @protosam. We’re working on making our cluster HIPAA complaint since we’re into healthcare domain, logging whatever the commands/action performed against the cluster is also part of HIPAA complaint rules for auditing purpose.

6

From my own experiences, you should consult whoever your auditor is about this. You need to explain the correct solution and show them that you implemented it.

Auditors for compliance don’t typically know our underlying systems. They normally just give out generic instructions. So you have to interpret those and fit those correctly in whatever infrastructure you’re working with.

If your auditor is not amicable to best practices and the justification you give, then you either have to implement audit logging on every user’s computer (which you should already have in place for hippa compliance) or you build a bastion with audit logging and force every user to kubectl from it (which sounds great until you start considering the logistics of the manifest files).

7

Makes sense @protosam, if nothing works out, my plan is to have a bastion in place with a private cluster that can be accessed only from bastion and log ssh sessions.
Now just trying to explore if there is a way that we can achieve it. Even I tried Falco which also provides deployment changes logs and some container runtime security. Will push everything to Elasticsearch and see if we have enough logs for auditing.