Launch configurations reference

neoaggelos · March 8, 2023, 6:25am

NOTE: Launch configurations are currently beta and will be released with MicroK8s 1.27. Until then, you can test them by installing MicroK8s from the latest/edge or the latest/edge/strict channel.

Launch configurations schema

The schema is defined in schema.go. For a full example config file, see full.yaml

Launch configurations version 0.1.0

---
# 'version' is the semantic version of the configuration file format.
version: 0.1.0

# 'extraSANs' is a list of extra Subject Alternate Names to add to the local API server.
extraSANs:
  - 10.10.10.10
  - microk8s.example.com

# 'extraKubeAPIServerArgs' is extra arguments to add to the local node kube-apiserver.
# Set a value to null to remove it from the arguments.
extraKubeAPIServerArgs:
  --authorization-mode: RBAC,Node
  --event-ttl: null

# 'extraKubeletServerArgs' is extra arguments to add to the local node kubelet.
# Set a value to null to remove it from the arguments.
extraKubeletArgs:
  --cluster-dns: 10.152.183.10

# 'extraKubeProxyArgs' is extra arguments to add to the local node kube-proxy.
# Set a value to null to remove it from the arguments.
extraKubeProxyArgs:
  --cluster-cidr: 10.1.0.0/16

# 'extraKubeControllerManagerArgs' is extra arguments to add to the local node kube-controller-manager.
# Set a value to null to remove it from the arguments.
extraKubeControllerManagerArgs:
  --leader-elect-lease-duration: 30s
  --leader-elect-renew-deadline: 15s

# 'extraKubeSchedulerArgs' is extra arguments to add to the local node kube-scheduler.
# Set a value to null to remove it from the arguments.
extraKubeSchedulerArgs:
  --leader-elect-lease-duration: 30s
  --leader-elect-renew-deadline: 15s

# 'extraContainerdArgs' is extra arguments to add to the local node containerd service.
# Set a value to null to remove it from the arguments.
extraContainerdArgs:
  -l: debug

# 'extraContainerdEnv' is extra environment variables (e.g. proxy configuration) for the local node containerd service.
# Set a value to null to remove it from the environment.
extraContainerdEnv:
  http_proxy: http://squid.internal:3128
  https_proxy: http://squid.internal:3128

# 'extraDqliteArgs' is extra arguments for the local node k8s-dqlite service.
# Set a value to null to remove it from the arguments.
extraDqliteArgs:
  --disk-mode: "true"

# 'extraDqliteEnv' is extra environment variables for the local node k8s-dqlite service.
# Set a value to null to remove it from the environment.
extraDqliteEnv:
  LIBRAFT_TRACE: "1"
  LIBDQLITE_TRACE: "1"

# 'addons' is a list of addons to enable or disable.
addons:
  - name: dns                               # 'name' is the name of the addon to enable.
  - name: gpu
    args: [--version=v22.9.1]               # 'args' is extra arguments that will be passed to the enable script.
                                            # refer to the documentation of each addon for more information.
  - name: registry
    disable: true                           # 'disable' should be set when the addon must be disabled instead.

# 'containerdRegistryConfigs' is used to configure registry mirrors. the key is the repository name (e.g. "docker.io")
# and the value will be written to the respective hosts.toml file
containerdRegistryConfigs:
  # Use `http://my.proxy:5000` as a DockerHub registry mirror.
  docker.io: |
    [host."http://my.proxy:5000"]
    capabilities = ["pull", "resolve"]

Examples

The following examples present launch configurations for common tasks. The examples are meant to be composable, you can mix and match as many of these configs as you want to suit your needs:

1. Enable DNS

Enable DNS addon, use host resolv.conf for upstream nameservers or fallback to 1.1.1.1.

# 01-dns.yaml
---
version: 0.1.0
addons:
  - name: dns

# These arguments will be set by the 'dns' addon. We set them manually to avoid
# unnecessary Kubernetes restarts while the cluster is bootstrapping.
extraKubeletArgs:
  --cluster-dns: 10.152.183.10
  --cluster-domain: cluster.local

2. Enable DNS, RBAC, Metrics-Server, Dashboard, Hostpath Storage, Ingress and Cert-Manager

# 02-setup.yaml
---
version: 0.1.0
addons:
  - name: dns
  - name: rbac
  - name: metrics-server
  - name: dashboard
  - name: hostpath-storage
  - name: ingress
  - name: cert-manager

# These arguments will be set by the 'dns' and 'rbac' addons. We set them manually to avoid
# unnecessary Kubernetes restarts while the cluster is bootstrapping.
extraKubeletArgs:
  --cluster-dns: 10.152.183.10
  --cluster-domain: cluster.local
extraKubeAPIServerArgs:
  --authorization-mode: RBAC,Node

3. Enable DNS with specific nameserver

Enable DNS addon and use a specific upstream nameserver (by specifying it as argument to the DNS addon):

# 03-dns.yaml
---
version: 0.1.0
addons:
  - name: dns
    args: [1.1.1.1, 8.8.8.8]

4. Configure private registry mirrors

Configure private registry mirrors for docker.io and registry.k8s.io. Requires that you have manually pushed required images to the registry mirrors, or that you have set them up as a pull-through cache:

# 04-registry-mirrors.yaml
---
version: 0.1.0
addons:
  - name: dns
containerdRegistryConfigs:
  docker.io: |
    [host."http://dockerhub.internal.mirror:15050"]
    capabilities = ["pull", "resolve"]
  registry.k8s.io: |
    [host."http://registryk8sio.internal.mirror:15050"]
    capabilities = ["pull", "resolve"]

5. Configure proxy for access to image registries

Containerd will use this proxy when pulling images from the upstream repositories:

# 05-containerd-proxy.yaml
---
version: 0.1.0
addons:
  - name: dns
extraContainerdEnv:
  http_proxy: http://squid.internal:3128
  https_proxy: http://squid.internal:3128
  no_proxy: 10.0.0.0/8,127.0.0.1,192.168.0.0/16,172.16.0.0/12

6. Configure dqlite disk-only operation mode

Use dqlite’s disk-only mode:

# 06-dqlite-disk-mode.yaml
---
version: 0.1.0
extraDqliteArgs:
  --disk-mode: "true"

7. Configure Subject Alternate Names for kube-apiserver certificates

Ensure the kube-apiserver can be accessed using the hostname k8s.cluster1.infra1.pod.

---
version: 0.1.0
extraSANs:
  - k8s.cluster1.infra1.prod

8. Configure Kubernetes services

Decrease lease times for kube-scheduler and kube-controller-manager from their default values. Also, tell kube-apiserver to prefer InternalIP when talking to the kubelets:

---
version: 0.1.0
extraKubeAPIServerArgs:
  --kubelet-preferred-address-types: InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP
extraKubeControllerManagerArgs:
  --leader-elect-lease-duration: 30s
  --leader-elect-renew-deadline: 15s
extraKubeSchedulerArgs:
  --leader-elect-lease-duration: 30s
  --leader-elect-renew-deadline: 15s

9. Configure host interfaces

Configure host interfaces used by Kubernetes services. In this example, the host uses 10.0.1.10 for control plane traffic, and 10.0.2.10 for NodePort services.

---
version: 0.1.0
extraKubeAPIServerArgs:
  --advertise-address: 10.0.1.10
extraKubeletArgs:
  --node-ip: 10.0.1.10
extraKubeProxyArgs:
  --nodeport-addresses: 10.0.2.10