Has anyone run into AWS API rate limit due to operating multiple small clusters in same AWS account?
What is the best practice?
Cluster information:
Kubernetes version: 1.11
Cloud being used: AWS
Host OS: Linux
AWS has API-call rate limits, which apply to your whole account. They aren’t published, and they vary based on AWS load.
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html
Thanks, so, Is it better to have fewer bigger K8s clusters than more smaller clusters to avoid exceeding those limits? How do you do team and work load isolation in such multi tenant K8s environment?
No :-/
Kubernetes tries to cope with that and react nicely. But of course that can still happen.
What limit are you hitting? What is the problem it causes?
As a very common alternative, every time more companies are using several AWS accounts and Amazon is making that use case more easy with time (you can have one account that it expenses are included in some other account that has the credit card, etc.). Not sure if this approach is good for your use case, though