What is Unbalanced IRQs Kubernetes Event?

General Discussions
1

We can see a new Kubernetes Event - Unbalanced IRQs - 5 IRQs with affinity to 1 CPUs.
Could you please share some insights on this event, its root cause, impact on the cluster and any solutions to resolve this warning?

Thanks for your time.

2

IRQs are how the kernel is notified of hardware events. This has nothing particular to do with Kubernetes.

3

We started getting this event a few days ago. I tried searching the keywords online (Google, Github Org, node-problem-detector repo, etc.) and got no results other than this thread. Any idea what this means? We are running v1.26.3 on AKS.

Screenshot from 2023-11-01 08-39-51
Screenshot from 2023-11-01 08-39-51584×813 40.6 KB

4

Same here, starting 1 or 2 weeks ago:
Reason: UnbalancedIRQs
Message: 9 IRQs with affinity to 1 CPUs
Source: irqbalance-problem-monitor NODENAME
Object: Node/aks-NODENAME

5

Possibly NPD started reporting this recently? Or maybe your OS was updated and the kernel is routing all IRQs to a single CPU?

6

We also started receiving these events in the node logs after upgrade AKS from 1.25.6 to 1.26.6.

Events:

|Type   | Reason       | Age                   | From                     | Message                       |
|-------|--------------|-----------------------|--------------------------|-------------------------------|
|Warning|UnbalancedIRQs|100s (x1930 over 6d16h)|irqbalance-problem-monitor|9 IRQs with affinity to 1 CPUs |
7

We faced the same issue with AKS.
In the cluster resource under “Resource Health” and then “Diagnose and solve Problems” the following was listed:

image
image1920×456 58 KB

So it seems like an AKS issue which could be fixed by updating the node images.