GitOps Toolkit Roadmap and the road to Flux v2

We are very happy to announce that we finalised the plans for the GitOps Toolkit and thus the road to Flux v2. We are especially pleased as we got great feedback and contributions from a wide range of people.

As you all know Flux v1 is a monolithic do-it-all operator, and the GitOps Toolkit separates the functionality into specialized controllers. Flux v2 will be a curated configuration of the GitOps Toolkit which you can simply consume using the tk command. Limiting the use of features and/or adding extensions on top of Flux has never been this easy.

You can review the roadmap in greater detail here. The big blocks we want to land are:

  • Feature parity with Flux v1 in read-only mode (80% completed). Once we’re done we will be able to ask people to transition over.
  • Image update capabilities (design in progress). This constitutes the biggest block of work still.
  • Helm Operator v2 (50% done).

We will move Flux v1 into maintenance mode now (no new features will be added). Once above work items are done, we will still support Flux v1 for 6 months - this will be a transition period for everyone to update their deployments.

If you immediately want to get started and check things out, that’s great. You can find more information here:

We believe we are building something extraordinary here - we will need your help to do it! We want the GitOps Toolkit to be a set of components that fit the bill for all GitOps needs. One example for how the components are meant to be reusable is Flagger: the team is looking at using the Toolkit’s notification-controller.

Here are some ways how you can help:

Please talk to us if you have any questions or feedback.

2 Likes

Cross-posted from https://www.weave.works/blog/flux-v2-september-update

The Road to Flux v2 - September update

This is the first blog post in a series which accompanies the development of the GitOps Toolkit on the road to delivering Flux v2 and more. We want to highlight what has been happening in development in the past weeks, who has contributed and what you can look forward to. And if you are curious, how to try it out for yourself.

It has been three weeks since the Flux community announced their plans for Flux v2 that has been rebuilt from scratch and based on top of the new GitOps Toolkit. If you haven’t heard the news yet, here’s what we are planning on doing:

  1. The GitOps functionality in Flux and Helm Operator is now delivered by individual components, to enable and support far more use-cases and to simplify development.
  2. Everything is driven by Custom Resources for more control and better observability.
  3. We built the GitOps Toolkit from the ground up incorporating the most requested features:
  • Support for multiple source Git repositories
  • Operational insight through health checks, events and alerts
  • Multi-tenancy capabilities, such as applying each source repository with its own set of permissions

In April this year we started with early experimentation for building a proof-of-concept, which was first shown at the GitOps Days virtual event in May, followed by a request-for-comments period and roadmap discussions in June. Since then the Flux community has come a long way.

Our highlight in this episode: the road to Helm Operator v2

Since the announcement, the team has made great strides, especially in the area of Helm: around 90% of all the work related to building the “Helm Operator v2” is done. The Helm Controller now has support for values from Secret and ConfigMap resources, and support for strategies for handling different kinds of failure. Its API package is also available as an independent Go Module. Failure handling is a big one. It means that you can extensively configure how to automatically remediate an installation, test, or upgrade failure. This includes actions that should be performed to remediate (rollback or uninstall), the number of retries, and even if the release has failed and retries have expired, you can debug the issue more easily. In addition, it is possible to ignore test failures if necessary (as opposed to remediate after a failed “helm test”).

Also the API to support Helm releases, based on source API, has been discussed and designed. In particular, providing values from sources and support for Helm charts from Git. So what’s left for Helm Controller to be on par with today’s Helm Operator? Only implementing support for referring to an alternative chart values file. And of course lots of testing. Before we tell the world they can transition to Helm Operator v2, we need to hear from current users and integrate their feedback.

If you are using Helm Operator right now, it would be fantastic if you could test the Helm Controller: simply check out https://toolkit.fluxcd.io/guides/helmreleases/ and give us feedback on how the journey went for you (find contact info at the bottom of this blog post).

In other news

Here’s a recap of what happened in the project overall in the past three weeks:

  • Support for HelmChart artifacts built from GitRepository sources.
  • Various bug fixes, e.g. Notification Controller: fix to the Prometheus scraping endpoint.
  • We were approached by members of the Grafana Tanka project who decided to call their binary “tk” so we decided to go with “gotk” from now on.
  • In the future we will also be able to enable ARM support; one blocker for this is that Flux shelled out to the kustomize binary, which has been fixed now.
  • Mozilla SOPS decryption support is coming!

More projects starting to look at using go-git-providers: eksctl, libgitops, and through libgitops: ignite, crdconv, cluster-api-provider-existinginfra and wksctl.

Since Flux become a CNCF Sandbox project about a year ago, we formally started the process to review the project with the CNCF - the PR is worth a read if you are interested in where the project stands as a whole and what happened in the last year. The amount of care and contributions our large community of users and contributors has shown is somewhat mind-boggling. We are certainly very pleased with what we have achieved.

Development-wise the focus for the next few weeks is going to be:

  • Figuring out the final form of Flux v2
  • The next big milestone: Bringing the image automation on par with Flux v1
  • Increasing general test coverage

If you want to help us shape the future of any of these, please reach out - we would love to hear from you.

Special thanks to our contributors - here’s what they landed:

  • Martin H Berwanger improved the gotk install script and automated the upgrades of controller component dependencies in the Toollkit.
  • Sean Eagan contributed the conditional remediation on failed actions to the helm-controller.
  • Philip Laine implemented notifications for Github commits.
  • We also had many folks jump into conceptional work in our Github Discussions:
    • Shane Sveller is looking into non-curl-bash installation methods
    • Moshe Immerman had a look at the image update automation design
    • Sean Eagan has been working on various parts of the Helm controller planning. Calle Petterson, Jonas Arneberg and Steve Hipwell as well.
    • Daniel Morgan in SOPS support and Philip Laine in e2e testing, exposing image fields in kustomization resources and a possible Job controller.

There is still lots where you can help shape the future of the GitOps Toolkit. Some of the important discussions are:

We are looking forward to working with you.

Here is how you can get involved: