Custom Autoscaling in Kubernetes

General Discussions
1

Hi, for the past few months I’ve been working on an open source project to let people write their own scaling logic for Kubernetes - based largely on the Horizontal Pod Autoscaler.

Here is the project repo: https://github.com/jthomperoo/custom-pod-autoscaler

Overview

Custom Pod Autoscalers

These scalers are called Custom Pod Autoscalers, which are basically a Pod running in the cluster managing resources such as deployments. The idea is that all interactions with the Kubernetes API should be abstracted away, with the logic that a developer would write being able to focus on the actual scaling mechanism.

Each Custom Pod Autoscaler has a base program, which handles all Kubernetes API interactions, and handles triggering the custom logic that the developer provides by calling it through a shell command with relevant information piped to it. The thinking behind using a shell command and piping relevant information to it is that practically any language could be used, for example Python scripts, entire Go programs etc; the only requirement is it must be triggerable by a shell command.

The user’s logic is split into two stages:

  • Metric gathering - generating, calculating or retrieving metrics, could be from the K8s metrics API, or from any other source.
  • Evaluating - taking these gathered metrics and making a decision on how many replicas a resource should have.

The custom logic receives piped in relevant info, such as Pod or Deployment JSON, and then calculates/retrieves values before outputting into stdout; which is read by the base program. If a non-zero exit code is returned it is assumed the metric gathering failed and an error is reported.

The base program then takes the evaluations that are calculated and interacts with the Kubernetes API to scale resources.

Custom Pod Autoscaler Operator

In order to correctly run these Custom Pod Autoscalers a number of Kubernetes resources are required to be provisioned, e.g. Roles, Service Accounts. To simplify this I’ve created an operator to make installing these really simple.

Here is the project repo for the operator: https://github.com/jthomperoo/custom-pod-autoscaler-operator

With this operator, you can just define a CustomPodAutoscaler in YAML to deploy it to your cluster.

apiVersion: custompodautoscaler.com/v1alpha1
kind: CustomPodAutoscaler
metadata:
  name: example-cpa
spec:
  template:
    spec:
      containers:
      - name: example-cpa
        image: example-cpa:latest
        imagePullPolicy: IfNotPresent
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: hello-kubernetes

Examples

Scale on Tweet

https://github.com/jthomperoo/custom-pod-autoscaler/tree/master/example/scale-on-tweet
This is a novelty example, showing an autoscaler that queries the Twitter API searching for tweets with a specific hashtag. Counting the number of tweets that contain ‘:+1:’ and the number of tweets that contain ‘:-1:’ and sets the number of replicas to the difference between number of tweets containing :+1: and :-1:.

This example was set up to show the flexiblity of Custom Pod Autoscalers, you can scale on anything - it also showcases the simplicity of the actual scaler, with a metric.py and evaluate.py that are quite readable and not dealing with K8s. For example the main function of metric.py is this:

def main():
    # Load twitter auth
    consumer_key = os.environ[CONSUMER_KEY_ENV]
    consumer_secret = os.environ[CONSUMER_SECRET_ENV]
    access_token = os.environ[ACCESS_TOKEN_ENV]
    access_token_secret = os.environ[ACCESS_TOKEN_SECRET_ENV]
    # Load watched hashtag
    hashtag = os.environ[HASHTAG_ENV]

    # Set up API
    api = twitter.Api(consumer_key=consumer_key,
                  consumer_secret=consumer_secret,
                  access_token_key=access_token,
                  access_token_secret=access_token_secret)

    # Get tweets with hashtag
    tweets = api.GetSearch(raw_query=f"q=%23{hashtag}&result_type=recent&count=100")

    # Count number of thumbs up and thumbs down
    num_up = 0
    num_down = 0
    for tweet in tweets:
        if "👍" in tweet.text:
            num_up += 1
            continue
        if "👎" in tweet.text:
            num_down += 1
            continue

    # Output number of thumbs up and down
    sys.stdout.write(json.dumps(
        {
            "up": num_up,
            "down": num_down
        }
    ))

Horizontal Pod Autoscaler

https://github.com/jthomperoo/horizontal-pod-autoscaler
This is the Horizontal Pod Autoscaler reimplemented as a Custom Pod Autoscaler, with a similar if not identical featureset. I’ve tested out a few metric types on this and they seemed to work, but this was by no means thorough so this probably has quite a few bugs in it.

This was set up mainly as a jumping point for other developers who maybe just would rather tweak the Horizontal Pod Autoscaler in some way.

Future

I’m looking to try to get some feedback if people think this would be useful, and if so am I pushing the project in the correct direction. I want to try to get a stabilised API that I could make a production ready release with, so I would really like feedback to try and catch some of my mistakes before I cut version 1.0.0. I’m also looking for feedback as to bits that I could explain more thoroughly/better, criticism is welcome!