It’s that time again! Kubernetes Office Hours is our monthly livestream where Kubernetes developers answer user questions live on the air. We’ll use this thread as a place to collect questions, so if you’re stuck then this would be a good time to ask for help!
We give out shirts! Though we’re in need of a resupply from the CNCF so there will be delay with these, but we’re working on it.
Join us tomorrow at 9am ET / 2pm UTC for our Office Hours, every skill level invited!
The panel was talking fast! Here are the notes and URLs from the show:
Question: We’re writing a controller with controller-runtime, and trying to use the Generation/ObservedGeneration pattern to avoid reconciling if there isn’t any change (not using the predicate provided by controller-runtime for that purpose yet though). My question is how can that work with the possibility of a stale cache? When we write the ObservedGeneration to the Status of our CR, it triggers another reconcile immediately, but in some cases, the cache is stale and the CR it "Get"s still has the old Status, and therefore the old ObservedGeneration. What is the recommended strategy of dealing with this? Thanks!
Person: Simone Baracchi
Question: I’d like to configure my small cluster as “highly available” with no single master / single point of failure and make the best use of all the cluster resources. My current plan is to make 3 nodes run as masters and be able to schedule pods on the masters. From my research the issues in doing so are 1) security issues about sensitive data on master which could be read from malicious pods and 2) pods competing for resources (especially in case of a node failure). I’m not too concerned about security atm, and I can think of limiting the max number of pods / resources used. Is there any other red flag in doing so?
Person: Jesper Berg Axelsen
Question: Is it possible to limit a service account to only have rights to create, edit and delete custom resource definitions that are related to a certain namespace? Since CRDs are not namespaced, I only see the option to give my service account rights to create, edit and delete all CRDs on the cluster. For the system we are creating, we do not see this as secure and would like to know if there is a way to limit our service account?
Question: I’m planning to install falco on our AWS nodes, what’s the best way to bring them up when using auto-scaling or adding new nodes to the pool.
Person: Ram Iyengar
Question: What are the general areas that K8s security is working on?
- community/README.md at master · kubernetes/community · GitHub
- community/RFP.md at master · kubernetes/community · GitHub
- Kubernetes Policy Comparison: OPA/Gatekeeper vs Kyverno | Neon Mirrors
- k8s-labs/README.md at main · knabben/k8s-labs · GitHub
- February 2021 Eastern Canadian CNCF Meetup: CKS and Hierarchical Namespaces - YouTube
- GitHub - walidshaari/Certified-Kubernetes-Security-Specialist: Curated resources help you prepare for the CNCF/Linux Foundation CKS 2021 "Kubernetes Certified Security Specialist" Certification exam. Please provide feedback or requests by raising issues, or making a pull request. All feedback for improvements are welcome. thank you.
Question: When I delete pvs (retention policy is Retain), pv remains in the system which is expected but when I delete that pv, I expect the underlying Volume to be deleted on The Cloud provider but that doesn’t happen.
So does this mean that one has to manually go and cleanup all the volumes which remains in the cloud due to this behaviour. One has to take care of this otherwise it will increase the unnecessary cost.
I wish to create a feature request to enable the deletion of underlying volume on deletion of pv. What are your views on this. Am I missing something here ? I’m on EKS.
Question: I am completely new to this and am investigating whether it is worthwhile for my company to move to Kubernetes. We run several (smaller) websites and some of them get a lot of traffic after a social media post. My main concern is abstracting the manual provisioning of VM’s because it is burdensome to document the configuration. I think it would be easier with containers. Any general advice on when it’s worth moving to k8s?
- On Demand Webinar: How to Pitch Kubernetes to Management
Question: Hi are there any tools for finding and deleting dangled RBAC rules in kubernetes mainly onprem
Doesn’t look like there’s any.
IaC would help here
Other tools to look at for RBAC reviews:
- GitHub - corneliusweig/rakkess: Review Access - kubectl plugin to show an access matrix for k8s server resources
- GitHub - FairwindsOps/rbac-lookup: Easily find roles and cluster roles attached to any user, service account, or group name in your Kubernetes cluster
Person: Mostafa Elmenbawy
How can I use a GPU by multiple pods? i.e request fraction of a GPU from a pod
- GitHub - AliyunContainerService/gpushare-device-plugin: GPU Sharing Device Plugin for Kubernetes Cluster
- GitHub - AliyunContainerService/gpushare-scheduler-extender: GPU Sharing Scheduler for Kubernetes Cluster
Link: Container Startup Sequence
Question: Hello, i would like to ask if containers are started in the order they are written in a statefulset manifest file?
“Then the kubelet runs the Pod’s init containers in the order they appear in the Pod’s spec.”
(Init Containers | Kubernetes)
Is this true for non-init containers?
Link: There a way to validate my yaml on Kubernetes 1.20?
Question: There a way to identify which part of my yaml is not compatible with Kubernetes 1.20 ?
Question: Is there any tutorial where i can learn Kubernetes from scratch?
Answer: There are many options.
- Kubernetes the hard-way GitHub - kelseyhightower/kubernetes-the-hard-way: Bootstrap Kubernetes the hard way on Google Cloud Platform. No scripts.
- Tutorials | Kubernetes
- GitHub - ramitsurana/awesome-kubernetes: A curated list for awesome kubernetes sources
- Creating a cluster with kubeadm | Kubernetes
- Scalable Microservices with Kubernetes
- Introduction to Kubernetes (LFS158) - Linux Foundation - Training
- Scalable Microservices with Kubernetes
- Play with KIND: https://kind.sigs.k8s.io/
- Kubernetes Tutorial for Beginners [FULL COURSE in 4 Hours] - YouTube
- Ellen Korbes’ tooling talk: GOTO 2019 • Kubernetes Day 3: The State of Kubernetes Development Tooling • Ellen Körbes - YouTube
Answer: How to manage dns i.e., create, destroy sub domains etc… in route53 for public facing applications running on eks?
- GitHub - kubernetes-sigs/external-dns: Configure external DNS servers (AWS Route53, Google CloudDNS and others) for Kubernetes Ingresses and Services
- Yogi: I use a simpler technique.
Create an ingress controller. Gives me an LB Address.
Create 2 route 53 entries.
*.cluster.domain.com > IP
cluster.domain.com > IP
- Brian Davis: kube2iam to use annotations for isolated roles
- gatekeeper-library/library/general/uniqueingresshost at master · open-policy-agent/gatekeeper-library · GitHub
- GitHub - open-policy-agent/gatekeeper-library: The OPA Gatekeeper policy library.
- library/kubernetes at master · open-policy-agent/library · GitHub
- Building a Kubernetes Mutating Admission Webhook | by Adil H | Medium
- Stream this week! Introduction to Kyverno | Rawkode Live - YouTube
- OPA Mutations: Using Open Policy Agent to Meet Evolving Policy Requirements - Jeremy Rickard, VMware - YouTube
Person: Ruaridh Angus
Question: Multi-Cluster service discovery between EKS and Bare Metal, currently running Consul but we have found some undesirable behaviour with health checking while using catalogue sync. Do we have a “preferred” / native solutions that are simple(well no more complicated than consul) and don’t involve using headless services and then loosing load balancing like coreDNS kubernetai.