Kubernetes Community Meeting Notes

August 22, 2019

August 29, 2019

  • Moderators: Dawn Foster [Pivotal/ContribEx]
  • Note Taker: Craig Peters [Microsoft/SIG-x]
    • Subscribe to this thread to get these notes in your inbox
  • [ 0:00 ]** Release Updates**
    • Current Release Development Cycle [Lachlan Evenson - Release Manager]
    • 1.16 Upcoming Milestones
      • 8/29 - 1.16 Code Freeze - label your PRs appropriately! The backlog is big and you don’t want to miss the train
      • 9/3 - Docs PRs ready for review - next Tuesday
      • 9/4 - 1.16.0-beta.2
    • Patch Release Updates.
    • Reminder these pending dates are announced on:
  • [ 0:00 ]** Demo **-- Ignite [@luxas] - confirmed
    • Slides
    • Simplified firecracker UX using the GitOps management model
    • Questions
      • Use of Virtual Kubelet vs CRI (easier development and UX), and
      • Difference from kata + kubevirt (full VMs instead of containers)
  • [ 0:00 ] SIG Updates
  • [ 0:00 ] :mega:Announcements :mega:

September 5, 2019

Thanks to @markyjackson for helping on Jenkins credential issue and sharing his thoughts on Jenkins automation

September 12, 2019

bentheelder:fire: - shoutout to @liggitt for reviewing all of the things

1 Like

September 26, 2019

  • Moderators: Tim Pepper [VMware/SIG Release]
  • Note Taker: Lachlan Evenson [Microsoft/SIG PM]
    • Subscribe to this thread to get these notes in your inbox
  • [ 0:01 ] Demo – Octant: A web-based, highly extensible platform for developers to better understand the complexity of Kubernetes cluster [Bryan Liles, @bryanl; Wayne Witzel, @wwitzel3]
    • Web-based, but runs local, using your credentials (simplifies security)
    • Demo application troubleshooting via the Octant UI
      • Web app working
      • Kubectl apply updated app
      • Web app no longer working
      • Use Octact to determine the cause
    • Introduces the concept of “Application” which is a set of consistent labels “app.kubernetes.io/name:httpbin”
    • Visualization of dependency graph between Kubernetes resources. Detects that the Ingress is pointing to an invalid backend
    • Drill down into service via the visualization graph and we notice that are no endpoints.
    • Determine that it’s a bad selector and update and check that the graph is green again.
    • If you’re on a Mac you can install via brew install octant

[ 0:14 ] Release Updates

  • 1.17 Release Development Cycle [Guinevere Saenger - Release Manager]
    • Week 1
    • Shadow selection happening (application deadline yesterday)
    • Please be aware that this is a short release
    • Enhancements freeze 10/15 5pm Pacific
  • Patch Release Updates
    • UPCOMING RELEASE SCHEDULE link
    • Patch Release Cherry-picks deadline Target date
    • 1.16.2 2019-10-11 2019-10-15
    • 1.16.1 2019-09-27 2019-10-02
    • 1.15.5 2019-10-11 2019-10-15
    • 1.14.8 2019-10-11 2019-10-15
    • 1.13.12 2019-10-11 2019-10-15 (final release of 1.13)
    • …as always subject to change for critical-urgent security issues

[ 0:17 ] Contributor Tip of the Week [Bob Killen]

[ 0:19 ] SIG Updates

[ 0:43 ] :mega:Announcements :mega:

1 Like

October 3, 2019

  • Moderators: Jonas Rosland [VMware/SIG Contribex]
  • Note Taker: First Last [Company/SIG]
    • Subscribe to this thread to get these notes in your inbox
  • [ 0:00 ]** Steering Committee Election Results **[Dims]
    • The following candidates will be joining @dims, @tstclair, and @spiffxp on the Steering Committee (in github handle order):
      • Christoph Blecker (@cblecker), Red Hat
      • Derek Carr (@derekwaynecarr), Red Hat
      • Nikhita Raghunath (@nikhita), Loodse
      • Paris Pittman (@parispittman), Google
    • See the blog post for more information
  • [ 0:00 ]** Release Updates**
    • Current Release Development Cycle [Guinevere Saenger - Release Manager]
      • We’re in Week 2! Shadow selection is 99% complete - congratulations and thanks to all of our hardworking team members
      • Enhancements Freeze is 15 October!
      • 1.17.0-alpha-1 was released yesterday
      • Next alpha scheduled for 15 October
    • Patch Release Updates
      • 1.16.1 released 1 October
      • Next patch releases scheduled for 15 October
      • y.x
  • [ 0:00 ] **Contributor Tip of the Week **[First Last]
    • A fun graph, contribex info, CI tips, etc.
    • [Link to a chart, a guide, a tool, etc]
    • Reach out to #sig-contribex in slack if there is no tip on the agenda yet. Backlog is pinned to the chat.
  • [ 0:00 ] SIG Updates
  • [ 0:00 ] :mega:Announcements :mega:
    • **:clap: **Shoutouts this week (Check in #shoutouts on slack) :clap:
    • tpepper:
      • shoutout to @nikhita for a PR description and commit messages in https://github.com/kubernetes/kubernetes/pull/82410 which makes a potentially daunting code review MUCH easier, and to @liggitt for similarly making the cherry-pick review MUCH easier with a stellar PR description text. Super time saving when there’s a diffstat of “+2,537 −59” but the “why” text focuses the reviewer in on two key lines of code and the associated bugs tracking the problem report.
    • jdetiber:
      • Shoutout to @dims for building out the e2e conformance tests using Cluster API and the GCP Provider

October 10, 2019

  • Moderators: Marky Jackson [ Sysdig/SIG Contribex]
  • Note Taker: Bob Killen
  • [ 0:00 ]** Release Updates**
    • Current Release Development Cycle [Guinevere Saenger - Release Manager]
      • We’re in Week 3…
      • Enhancement Freeze is next Tuesday (Oct. 15). Enhancements must be in by 5PM PT.
      • 1.17.0-alpha.2 scheduled release Tuesday Oct.15
    • Patch Release Updates
      • 1.16.1 released 1 October
      • Next patch releases (all branches) scheduled for 15 October
      • LAST release of 1.13.x
  • [ 0:00 ]** SIG Updates**
    • WG Security Audit [Jay Beale]
      • Slides: https://docs.google.com/presentation/d/1yKjbvFqU0xp3wq0wY9Qu99WNA8FRGDGkaH5nHoCKxVM/edit#slide=id.g401c104a3c_0_0
      • What we did last cycle
        • Led the first in a series of Kubernetes security audits
          • Choose vendors
          • Gave direction to focus effort
          • Participated in the threat modeling work that will be used for future releases of Kubernetes
          • Performed technical editing on the report
          • Worked on producing reusable artifacts
        • Complementary efforts to the bug bounty program
        • Threat model breakdown
          • Focus on 8 critical components
            • Kube-apiserver
            • Etcd
            • Kube-scheduler
            • Kube-controller-manager
            • Cloud-controller-manager
            • Kubelet
            • Kube-proxy
            • Container Runtime Interface
        • Threat model highlighted recommendations
          • Provide auditing information in a unified fashion to allow a trace of the user’s actions through the system
          • Warn users who configure a security control that will not be enforced
            • Network policies and pod security policies can silently fail.
          • Require transport encryption w/cert verification
            • Multiple components use http
            • Multiple components elect not to verify cert validity
          • Prevent node compromises from leading to cluster-compromises
            • Host access gives access to cli arguments, logs etc
          • Separate privilege levels among controllers
        • Vulnerability research during cycle
          • Discovered 37 vulnerabilities
        • Vulnerability highlights
          • Non authenticated HTTPS connections
          • Cert revocation unsupported
          • PSP Bypass (hostPath va PVs)
          • TOCTOU Race condition in Kubelet
          • Kubectl cp directory traversal
          • System logs containing secrets
        • Recommendation Highlights
          • Replace the many cases of logic reimplementation with central libraries
          • Ease security configuration (particularly defaults)
          • Improve code documentation around external dependencies
          • Continue development of security features
        • Security Audit report [link from report in k/community]
      • Next cycle:
        • Plan next security audit
        • Move towards more secure defaults
    • SIG Testing [fejta]
      • https://docs.google.com/document/d/1uTcLhxM2HwDgtGOiIvlFfRWzQDTvii6qd_XASAubHlk/edit?ts=5d9e6825
      • Last Cycle
        • Testgrid configs now live alongside their associated prow jobs
        • Automated the creation of jobs for the test-infra release team role
        • Deployed new and improved monitoring/alerting stack (monitoring.prow.k8s.io)
        • Reusable verify checks in bazel rules
        • KinD
          • Smaller images from providerless kubernetes builds
          • Release blocking IPv4 and IPv6 test coverage
          • Provides 75% of pull-kubernetes-e2e-gce coverage without any cloud resources
        • TestGrid partially open sourced
      • Next Cycle
        • Establish test-infra SLOs
        • Improve test-infra alerting to better detect and recover from outages
        • Make KinD a blocking presubmit in k/k
        • Automate image pushing on merge with a git-ops based promotion to prod method (working with #wg-k8s-infra)
        • Help repos with preexisting bazel rules adopt reusable verify checks.
        • Move prow out of test-ifnra into its own repo
        • Enable in repo prowjob configurations
      • How these upcoming changes affect you
        • Help define more reusable verify checks
        • Start thinking about how/whether your sig can move cloud provider dependencies out of k/k testing to release blocking postsubmits
    • [ 0:00 ] :mega:Announcements :mega:
      • Announcement Foo #1
      • **:clap: **Shoutouts this week (Check in #shoutouts on slack) :clap:
        • @jdetiber** **gave a shout to @dims for building out the e2e conformance tests using Cluster API and the GCP Provider
        • @mrbobbytables gave a shoutout to the other Steering Election committee officials @briangrant @castrojo @ihor.dvoretskyi for putting in the work to make this year’s election possible!
        • @ihor.dvoretskyi gave a huge SHOUTOUT to @mrbobbytables - another election official!
        • @cblecker gave a** **shout out to @bentheelder and @krzyzacy for late night debugging on GCE test infra failures

October 17, 2019

  • Moderators: Jorge Castro [VMware/SIG Contributor Experience]
    • No video available, Jorge hit the wrong button on OBS. :frowning:
    • Subscribe to this thread to get these notes in your inbox
  • [ 0:00 ] Release Updates
    • Current Release Development Cycle [Guinevere Saenger - Release Manager]
      • Enhancements Freeze was this past Tuesday, 15 October
      • Two exceptions filed
      • We have 44 enhancements tracked: alpha: 11, beta: 13, stable: 20
      • 1.17.0-alpha.2 released on Oct.15
      • 1.17.0-alpha.3 planned for Oct.22
    • Patch Release Updates
      • All branches released 15 October
  • [ 0:00 ] SIG Updates
  • [ 0:00 ] :mega:Announcements :mega:
    • Don’t forget to register for the contributor summit!
    • **:clap: **Shoutouts this week (Check in #shoutouts on slack) :clap:
      • **@jdetiber - **shoutout to @cblecker for adding a /honk command to prow
      • @gsaenger - shoutout to @markyjackson for being such a friendly community meeting host!
    • SIG Usability, WG Apply, and WG Machine Learning will be giving updates next week!

October 24, 2019

  • Moderators: Jonas Rosland [VMware/SIG Contributor Experience/Release]
  • Note Taker: Thiscould B. You [Company/SIG]
    • Subscribe to this thread to get these notes in your inbox
  • [ 0:00 ]** Release Updates**
    • Current Release Development Cycle [Guinevere Saenger - Release Manager]
      • 1/17.alpha.3 released this Tuesday 10/22
      • All Enhancement exceptions are merged and tracked
      • Begin turnaround for release branch creation next week: removal of 1.13 jobs, create 1.17 jobs, create 1.17 release branch, cut the first 1.17 beta
      • Lots of work from SIG scalability and the CI Signal team to capture scale job flakes early and find causes
    • Patch Release Updates
  • [ 0:00 ] SIG Updates
    • SIG Usability [Tasha Drew @tasha]
    • WG Multitenancy [Tasha Drew @tasha]
    • WG Apply Working Group [Jenny Buckley @jennybuckley]
      • Slides
    • WG Machine Learning [punt till next week]
  • [ 0:00 ] :mega:Announcements :mega:

October 31, 2019

  • Moderators: Marky’s assistant Hammy :wink: [SIG Contributor Experience/Release]

  • Note Taker: Chris Short

  • [ 0:00 ] Release Updates [Guinevere Saenger - Release Lead]

    • 1.17.0-beta.0 released this Tuesday 10/29
    • 1.17 release branch created
      • All changes to master will be fast forwarded nightly into the 1.17 branch
    • CODE FREEZE IS COMING NOVEMBER 14
      • after Code Freeze, all approved enhancements work will need to follow cherry-pick process to be merged into the 1.17 branch
    • 1.13 jobs are being removed
  • Patch Release Updates

  • [ 0:00 ] SIG Updates

    • SIG Release [Stephen Augustus]
      • Improved feedback loops between SIG Release and SIG Scalability
      • Emeritus advisor is awesome
      • More diversity of all kinds in the Release Teams
      • Improvements in automation across the board
      • SIG Release needs more shadows
      • People are improving test coverage on their features
      • Release Engineering subproject has started in earnest
      • Test cleanup and deletion continues
      • Release Managers Group
      • Release Engineering
        • Onboarding process improvements
        • Wiring Release Engineering jobs in CI
        • Doc cleanups
        • Working on getting staging/release process into CI
        • Viewer access to GCP
        • k/release tooling is getting rewritten in Go and one tool has already been deployed
        • deb/rpm packaging tools are being built and awesome-ified
        • Hyperkube out-of-tree in progress
        • Codebase walkthroughs!!!
      • Watch for announcements
      • Pay attention to CI Signal
      • Be mindful of 1.17 schedule dates
      • We’ll be at KubeCon!
  • [ 0:00 ] :mega:Announcements :mega:

    • Don’t forget to register for the contributor summit!
    • :clap: Shoutouts this week (Check in #shoutouts on slack) :clap:
      • @dims gave a shoutout to @bartsmykla for setting up / running the on-boarding call for 70+ folks for wg-k8s-infra
      • Daniel Lipovetsky [@dlipovetaky] gave a shoutout saying,Thank you, thank you, thank you to @neolit123 [Lubomir Ivanov] for always taking the time to help and mentor. You have been there for me and for many others on what seems like everywhere from k/k, to kubeadm, to docs, and everything in between.

@markyjackson gave a shout out to @gsaenger @chrisshort and @rael for getting together to make the NCW awesome and for being such fine peoples to work with

1 Like

November 7, 2019

  • Moderators: Marky Jackson [Sysdig/SIG Contributor Experience/SIG Release]
  • Note Taker: [Jorge Castro/SIG Contributor Experience/VMware]
  • [ 0:00 ] Release Updates [Guinevere Saenger - Release Lead]
    • 1.17 release

      • “Calm before the storm” - KubeCon prep, lots of meetings
      • Everyone filing an enhancement MUST file a docs PR for it by TOMORROW
      • Code freeze next week, 14 Nov, everything afterwards will be a cherry pick
      • This tuesday, first beta of 1.17
    • Patch releases (schedule):

      • Cherry pick deadline tomorrow, Nov. 8 ahead of:
      • Release target Wed. Nov. 13
  • [ 0:00 ] SIG Updates
  • [ 0:00 ] :mega:Announcements :mega:
    • This is the last community meeting until December 5th
    • Happy Kubecon and happy thanksgiving
    • Don’t forget to register for the contributor summit!
    • :clap: Shoutouts this week (Check in #shoutouts on slack) :clap:
      • Chris Short gave a huge shoutout to @castrojo and @jeefy for getting me all set to stream community meetings. So helpful and kind (even when I forget things)!
      • Chris Blecker gave a shoutout to @liggitt and @bentheelder for their help in getting us upgraded to go1.13. It was a huge effort!
      • Paris gave a shoutout to everyone on the kubecon planning stretch especially the wonderful contributor summit events team

Sorry this one is late folks, the meeting has been on hiatus due to KubeCon and the holidays. The next meeting is on January 16.

December 5, 2019

(Recording not yet available)

  • Moderators: Jeffrey Sica [Red Hat, SIG-Contribex/Release/UI]
  • Note Taker: Jordan Liggitt / Bob Killen
  • [ 0:00 ]** Release Updates **[Guinevere Saenger - Release Lead]
    • 1.17 release
      • 1.17.0 targeting Monday, December 9th
      • Generally looking good, might have one bugfix in progress
      • Primary need is for SIG review of release notes
    • Patch releases (schedule):
      • Cherry pick deadline tomorrow, Dec. 6 ahead of:
      • Release target Wed. Dec. 11
  • [ 0:00 ] SIG Updates
  • [ 0:00 ] :mega:Announcements :mega:
    • One more Community Meeting before EOY!
    • Kubecon EU CFPs closed YESTERDAY GOOD LUCK
    • **:clap: **Shoutouts this week (Check in #shoutouts on slack) :clap:
      • Jeremy - Shoutout to @bentheelder for the real time kind troubleshooting for the new contributor workshop
      • Elana - shout out to @jeefy and @mrbobbytables for the best goose game ever. Honk.
      • Ben - Shoutout to @cblecker for adding #kind to homebrew! Thank you Christoph!
      • Paris - Shoutout to the fabulous kubernetes contributor summit team! Thanks for making the show in San Diego a memorable one. Can’t wait to see what’s next for Amsterdam!

January 16, 2019

NOTE: This meeting is now monthly!

  • Moderators: Laura Santamaria [LogDNA/SIG Contribex]
  • Note Taker: Bob Killen [University of Michigan/Contribex]
    • Subscribe to this thread to get these notes in your inbox
  • [ 0:00 ]** Release Updates **
    • Current Release Development Cycle [Bob Killen]
      • Tuesday, January 28: Week 4 - Enhancements Freeze
        • Implementable state
        • Have a test plan
      • Thursday, March 05: Week 9 - Code Freeze
      • Monday, March 16: Week 11 - Docs must be completed and reviewed
      • Tuesday, March 24: Week 12 - Kubernetes v1.18.0 released
    • Patch Release Updates https://git.k8s.io/sig-release/releases/patch-releases.md
      • 1.17.1 released Jan. 14
      • 1.16.5 coming today Jan. 16
      • 1.15.8 coming today Jan. 16
      • 1.14.11 coming today Jan.16 (to fix an upgrade scenario for 1.15)
      • A series of bugs have been identified in how the next beta tag is applied on these branches. For example when “v1.17.1” is tagged and released we also mark the branch with a tag “v1.17.2-beta.0". The bugs root cause goes back many years in the design and implementation of the “anago” tool used to build and release, but are partially corrected now. A complete fix likely will come first at the point we replace the “anago” tool.
      • Next patch releases target Feb. 11 (see: https://github.com/kubernetes/sig-release/pull/954)
  • [ 0:00 ] SIG Updates
    • SIG Leads, check out this set of recommended topics to cover during your update. Slide template if you need it, please also check the SIG Update Schedule!
    • SIG Cloud Provider [Walter Fender]
      • Slides
      • Promoted Node Zone/Region Topology Labels to GA
      • Upcoming Cycles
        • API Server Network Proxy alpa with goal to promote to GA in the upcoming cycles
        • Extract cloud provider dependencies from the core repo
        • Generate a controller migration lock mechanism for moving controllers safely between controller managers
        • Better support for providerless builds for cloud providers who are working out of tree
        • Improve the tooling/documentation around cloud controller manager and per cloud repos
        • Targeting removal of in-tree cloud providers by the 1.21 release
      • What impacts you:
        • In-tree cloud provider e2e tests are strong test signal, need to figure out how best to transition to out-of-tree
      • New KEP template to add new cloud providers.
      • Cloud Provider Extraction WG
        • Slated for extraction with 1.21
        • Cloud controller manager is green
    • SIG Autoscaling [Marcin Wielgus]
      • Slides
      • Cluster Autoscaler:
        • Switching from using raw scheduler predicates to Scheduling Framework. This will improve behavior of CA in various corner cases related to zone-specific storage and affinity/anti-affinity.
        • Added support for Packet
        • Improved performance/scalablity.
      • Vertical Pod Autoscaler:
        • Graduating VPA api to GA soon
      • Horizontal Pod Autoscaler:
        • Expanded API to control how fast and how many pods are added on scale-up and scale-down.
        • Added support for scale to 0 (currently flag-gated).
    • SIG Scheduling [Abdullah Gharaibeh]
      • Slides
      • What we did last cycle:
        • Scheduling Framework has hit milestone 1
          • Finalized framework implementation
          • Wrapped existing predicates and priorities functions in plugins
          • Added a translation layer from predicate/priority “policies” into Plugin configurations
        • Performance improvements
        • Improved observability: new latency, traffic and saturation metrics
          • Scheduling latency
        • Features graduated to GA in 1.17
          • Schedule DaemonSet Pods
          • Taint nodes by condition
        • Performance Improvements
          • Large Scale Clusters
          • 4x improvement in prefered pod affinity
          • Scheduling latency
          • How many pods via qued
      • Plans for upcoming cycles
      • Leadership position changes
        • Bobby Salamat stepped down as co-chair
        • Abdullah Gharaibeh new sig co-chair
    • SIG Scalability [Matt Matejczyk]
      • Slides
      • What we did last cycle
        • Improved Scalability and Performance Tess
          • Add support for more kubernetes concepts such as DaemonSets, StatefulSets, Secrets etc.
          • ClusterLoader2:
            • improved testsuite
            • Better crashloop detection
            • HA support
          • Build more scale tests into the release branches
          • Pod throughput tests (containerD vs Docker)
        • GuardingAgainst Performance Regressions
        • Performance Improvements
          • Watch Serialization Mechanism Improvements
          • Core Components Improvements:
            • NodeLifeCycleController
            • GC Controller
            • TaintManager
          • Watch Bookmarks went to GA
          • KEP for immutable secrets
      • Plans for upcoming cycles
        • Kubernetes Scalability Definition
          • Finalizing existing WIP scalability SLI/SLOs
          • Updating scalability envelope (thresholds)
          • Work on hardening and extending the scalability definition
        • Scalability & Performance Tests
          • Covering more Kubernetes concepts
          • Work on Kubemark v2: better cluster simulations
          • Add other tests: HA, upgrade, chaos etc
        • Bottleneck Detection & Performance Improvements
        • How these plans affect you
          • Scalability approval process
            • Will need to work with KEP owners to validate new features
          • Extending SLI/SLO Coverage
            • We’ll be reaching out to help us understand what is important to the users and community
          • Notable Regressions
            • Kubernetes v1.17.0 is vulnerable to #86483 that can break large clusters on master restart
  • [ 0:00 ] :mega:Announcements :mega:
    • Contributor Survey: https://www.surveymonkey.com/r/VYRJZ5G
    • Let SIG Contribex know if this new format worked for you by pinging us in Slack.
    • **:clap: **Shoutouts this month (Check in #shoutouts on slack) :clap:
      • Rawkode - Awesome props to @alculquicondor for jumping in at very late notice and getting us help with the release blogs for 1.17 :tada:
      • Sascha - Big pre-release shoutout to @macintoshprime regarding his release notes efforts! That’s a lot of work, big kudos to you and your team!
      • Zacharysarah - Shoutouts to @mrbobbytables and @gsaenger today for resolving a particularly thorny docs release PR!
      • Gsaenger - And @Damini Satya !
      • Nikhita - shoutout to @liggitt @sttts and @dims for tirelessly going through the back and forth the past week to get v0.17.0 tags shipped for published (staging) repos :tada:
      • @vincepri - Shoutout to @ncdc for the great high quality effort to improve Cluster API documentation book!
      • Bentheelder - shoutout to @timothysc for all your work in sig-testing, particularly in #testing-commons, and for your leadership in stepping down when you needed to
      • Bentheelder - shoutout to @yasker for all the help and patience with PVCs in sigs.k8s.io/kind and your work on github.com/rancher/local-path-provisioner, looking forward to github.com/kubernetes-sigs/kind/pull/1157
      • Paris - shouts to @jberkus @idealhack @cblecker @maria @markyjackson @mrbobbytables @spzala and many others in contribex for their thoughtful review of the upcoming contributor experience survey
      • Paris - shoutouts to the kubernetes blog team (#sig-docs-blog) for all of their work reviewing PRs and working with contributors on that workflow so our end users and other community members can have great content on the blog.
      • Markyjackson - Shout out to @mrbobbytables for patiently helping me fix a git problem. Really appreciate you
      • Nimbinatus/Laura - Shoutouts to @castrojo and @marky.jackson for all the help getting up and running for hosting my first community meeting today! Appreciate all y’all do
      • Jorge Castro - Huge shoutout to @parispittman for 2 years of service as cochair of SIG Contributor Experience!
1 Like