Hi there,
I have a question regarding the general traffic shaping best practices involving kubernetes. Normally, when we deploy the kubernetes cluster for our internal micro-service architecture, there are other peripheral services including storage, monitoring, etc around, mixing up all the traffics in a typical DC setup. Due to the nature of network, there could always be congestion somewhere in the system. To priorities some traffics, we need to have some way to tag the packets originated from certain pods/deployments so that it will less likely be dropped when it flows through a congested link.
I saw there is a supported method to add annotations to limit the pods ingress and egress bandwidth, which I have tested and worked well for single pod bandwidth limiting. However, this functionality is kind of limited when we need to handle multiple pods or prioritize traffic between kubernetes and non-kubernetes traffic. What I hope to have is a collaborative way between kubernetes and the general network switching platform to control the bandwidth. I know generally we could mark tos on ip headers and let switch to do the QoS work, but I did not find any reference or kubernetes support on it. It would be great if someone could share the ideas on this front, or any other approaches to gracefully solve this problem.
Thanks a lot.
Jianan.