[Question] Pod cgroup metrics from cadvisor in k8s v1.12

General Discussions
1

Hi,

Like this issue https://github.com/google/cadvisor/issues/2092 described, I upgraded to k8s v1.12 and found pod cgroup metrics which I didn’t see in v1.10 or earlier version. This change broke my prometheus queries like sum(container_memory_rss) by (pod_name) .
I wonder if anyone has the same problem with me or knows the reason.

Thanks,
Tong

2

In Kubernetes 1.10, you should use label_replace

 sum by(pod, namespace, kubernetes_io_hostname) (label_replace(container_memory_rss{pod_name!=""},
  "pod", "$1", "pod_name", "(.*)"))

Screenshot%20from%202018-12-14%2017-43-05
Screenshot from 2018-12-14 17-43-05.png876×85 7.72 KB

Since Kubernetes v1.12 add redundant metrics, you need filter them.

 sum by(pod, namespace, kubernetes_io_hostname) (label_replace(container_memory_rss{pod_name!="",container_name=~".+"},
  "pod", "$1", "pod_name", "(.*)"))

Hope it helps you!

1 Like