Office Hours for April 2020

General Discussions
1

Hear ye, hear ye! It’s almost the third Wednesday of the month, that means it’s time for Office Hours! Our monthly livestream where we have a panel of Kubernetes experts answer questions from the audience. We have two sessions for you this month! Feel free to hang out in #office-hours on slack.k8s.io and start queuing up your question or post it here. When the livecasts are done we’ll post the notes here!

EU Session

9am ET

West Coast US Session

9am PT

All expertise levels are welcome, and if you’re interested in more information or volunteering, check it out:

2

We are going live today in about an hour after this post!

3

EU Session Links

West Coast Session Links

Thanks to @samudrala for these notes!

  1. Question:

  2. Question:

    • Text:
      Do you have any suggestions on how to setup prometheus for cluster monitoring to keep it highly available? ie, would you put it in the same cluster, a separate cluster or an external vm?

    Do you have any resources on how to setup a “master” prometheus? I am not sure how to send from one prometheus to another.

  3. Question:

  4. Question:

  5. Question:

  6. Question:

  7. Question:

    • Text: I am currently setting up a bash script to setup our dev/staging GKE cluster so we can shut them down at night and restart them on mornings at will, but Im wondering if there’s any solution that exists that would allow me to bundle up all the resources needed, the cluster, the dns, the various public help charts (mongo, redis, etc…) with custom config and our own applications (happens to be also in custom helm charts) rather than a custom bash script? I know AWS has something called CloudFormation that might have allowed me to do this but we are using GKE. If many tools exists, which one you recommend?

    • Asker: Christian Roy

    • Answer: Terraform ,flux, Argo CD
      https://github.com/cloudposse/terraform-aws-eks-cluster
      https://eksctl.io/

  8. Question:

    • Text:earlier today, I was troubleshooting an issue and noticed that in a cluster there are deployment using pvc with a storage class local-storage and others with local-storage-local, however there is SC object when I do “get sc” that presents the storage class, how did that happen? also tried to check the node for annotation/labels related, there was one label localstorage=true but nothing corresponding in the deployment

    • Asker: Walid

    • Answer:

  9. Question:

    • Text: we are running machine learning model in kubernetes pod.if my pod is reached 100% pod resources automatically restarted but need to make some delay 10 mits to restart.how we can do?
    • Asker: devopsdymyr
    • Answer : set resource limits
      Configure Quality of Service for Pods - Kubernetes
      maybe pod prestop can help

  10. Question:

  11. Question:

    • Text: What’s the purpose of setting a CPU limit if the container might be allowed to exceed it and won’t be killed or is it just to override the Limit Range admission controller? How does k8s knows whether if it will allow or not, does it depends on the allocatable resources of the node?
    • Asker: Javier
    • Answer: Its a cgroup behavior
      -https://github.com/kubernetes-sigs/descheduler

  12. Question:

    • Text:What’s the best persistent storage for on-premise workloads? I have tried rook-ceph and Longhorn. What are you guys using?
    • Asker: Jawiz
    • Answer: rook, minio, portworx, NFS, and investigating vsphere(VSan) CSI
    • as a database geek, I do local volumes when I’m performance-constrained (and rely on DB replication), and when I’m not I do Ceph/Rook

URL’s:

https://infra.app/
https://kubecost.com/
https://github.com/kubectl-plus/kcf
https://github.com/rchakode/kube-opex-analytics
https://rancher.com/blog/2020/fleet-management-kubernetes/
https://github.com/getsentry/sentry
https://github.com/lensapp/lens
https://www.amazon.com/Kubernetes-Best-Practices-Blueprints-Applications-ebook/dp/B081J62KLW