Add a Windows worker node to MicroK8s

MicroK8s supports running Windows workloads. This guide will go through the process of joining a Windows Server 2019 node to an existing MicroK8s cluster running Calico.

Prerequisites

Export kubeconfig

To access the cluster, calicoctl will need a copy of the kubeconfig from MicroK8s. This can be made in any location - this example copies it to the default path:

mkdir -p ~/.kube
microk8s config > ~/.kube/config

Set affinity

In order for Windows pods to schedule, strict affinity must be set to true. This is required to prevent Linux nodes from borrowing IP addresses from Windows nodes. This can be set with the calicoctl binary. Be sure to point calicoctl to the Kubernetes API, rather than directly to Etcd.

DATASTORE_TYPE=kubernetes KUBECONFIG=~/.kube/config calicoctl ipam configure --strictaffinity=true

(replace the path for the kube config file if you saved it in a different location)

Gather the Kubernetes version

At a later stage, you will need to know the exact version of Kubernetes installed. This can be grabbed from MicroK8s.

microk8s kubectl get node $(hostname)

Note the output in the format X.X.X, e.g. 1.19.3.

Install components on the Windows node

You are now ready to install Calico onto the Windows node. This will also install Kubernetes components required for a working node.
All code snippets here should be run in PowerShell running as Administrator.

Create directory for Kubernetes

All Kubernetes components will be installed into this directory.

mkdir C:\k\

In here, place the kubeconfig file previously exported from MicroK8s. Be careful as some editors may try to append file extensions. You can test that it has been named correctly by trying to print the contents after you’ve saved it:

cat C:\k\config

Install

The Calico Project provides a helper script to fetch all the required binaries and services. Official documentation is available for this script, but the required steps are covered here.

First, download the script.

Invoke-WebRequest https://docs.projectcalico.org/scripts/install-calico-windows.ps1 -OutFile C:\install-calico-windows.ps1

You can then run the script with the required parameters. Change the -KubeVersion argument to the version noted earlier.

C:\install-calico-windows.ps1 -DownloadOnly yes -KubeVersion 1.19.3

Note: You can also explicitly set whether you are running Calico in vxlan or BGP mode by specifying this as an addidional argument, e.g.-CalicoBackend "vxlan"

Register the Calico services.

C:\CalicoWindows\install-calico.ps1

Then register the Kubernetes services so they come up with the node.

C:\CalicoWindows\kubernetes\install-kube-services.ps1

This script won’t start the Kubernetes services. Let’s do that now:

Start-Service kubelet
Start-Service kube-proxy

Cleanup

Remove the temporary files created in the previous steps:

rm C:\install-calico-windows.ps1
rm C:\calico-windows.zip

Finish

You can verify the node come up on the cluster by switching back to the MicroK8s node and running the command:

microk8s kubectl get no -o wide

Uninstalling

To uninstall the node components, simply run the downloaded script once more:

C:\install-calico-windows.ps1 -DownloadOnly yes -KubeVersion 1.19.3

This will indicate that a version is already installed and prompt if you want to remove it. Choose Y and the node components will be uninstalled

Upgrading

There is no specific upgrade path for a Windows node. If you have upgraded MicroK8s to a later version, and wish the Windows node to remian compatible, you should re-run the install script to uninstall the node components (see above), then run the installer again, indicating the new version of Kubernetes to use.

4 Likes

Thanks for the tutorial.
Was finally able to get the windows host joined into microk8s and deploy an example windows pod running mcr.microsoft.com/windows/servercore:1809 :slight_smile:

2 Likes

Hi @joedborg , @evilnick the official calico docs link is no longer available.

https://docs.projectcalico.org/getting-started/windows-calico/standard

Is it this one now?

Thanks

Thanks for noticing that. I believe you are right , it is now on https://docs.projectcalico.org/getting-started/windows-calico/quickstart

Oh, i updated the link in the doc too :wink:

Hey all, I am trying to set up a mk8s cluster with a windows worker node now and not getting any errors from the installer scripts, but once done my cluster never shows the Windows worker node. I reviewed some of the troubleshooting tips on the Canonical and Calico sites but can’t see what’s wrong here.

I am able to use the clusters kubeconfig file on the windows node to interact with the cluster via kubectl and I have reviewed the networking and believe all required ports are open correctly between worker node and control plane…

Do you guys have and troubleshooting suggestions for good troubleshooting articles or other places to look for assistance?

Hi, I can’t get this to work. I can get the windows node joined and can run a pod on it but there is no network connectivity. I didn’t get any errors during installation though I did have to change the default service cluster ip range to 10.152.183.0/24 to match microk8s and to set the service cluster ip of DNS to 10.152.183.10 to match the DNS Service. From the calico config file:

$env:K8S_SERVICE_CIDR = “10.152.183.0/24”
$env:DNS_NAME_SERVERS = “10.152.183.10”

I’m running microk8s on Ubuntu 20.04 (v1.20.6-34+e4abae43f6acde)
The windows node is Windows Server 1809 (Build 17763.1879)

I had similar problem. After a close look at the logs the installation script produces and the script itself I’ve found that the script fails to properly detect Calico backend - in my case bgp was used instead of vxlan. Specifying the backend explicitly resolved the problem for me:

 . C:\install-calico-windows.ps1 -DownloadOnly yes -CalicoBackend "vxlan"

thanks @zjklee for your fix. I will investigate though the Calico docs say the script should work with BGP or vxlan

I feel like I’m so close, but just not there yet. If someone can help me. I followed the described steps to a letter.
I managed to join Windows worker node. I can run windows containers there, but DNS isn’t working. If I use an IP address than pod from windows node can reach pod on linux node. But it doesn’t work with the service name. Same thing works for linux container. I tried to change DNS_NAME_SERVERS to 10.152.183.10 with no effect. I also tried -CalicoBackend “vxlan” suggestion with no luck. Is there anything else I can try?

I have missed ServiceCidr parameter (10.152.183.0/24)…now it’s working! This is so satisfying!

1 Like

Hi, I have gone thru the steps and set up 3 Microk8s nodes in Ubuntu and 1 node in MS Windows 2019. It’s kind of working that I could create and get the pod/deployment working in the windows node if it is a windows image. However, the cluster just randomly assigns the node regardless of the type of image windows / Linux. Any idea that I am going wrong please. Thanks!

Every node is labeled with the following

kubernetes.io/os and kubernetes.io/arch

Where kubernetes.io/os can be linux or windows. Using these labels you can use nodeSelector to make sure pods are scheduled into the right node.

For more details check this out.

1 Like

@balchua1 Thank you so much!

Hi there,

Not sure what happens but my hybrid cluster (ubuntu + windows) cannot deploy the pod to the windows node now. Any idea, please. Below is the message from describe pod. Thanks!

Warning FailedCreatePodSandBox 43s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox “27c91bdff828f4b4e6c07bd1ef2ae67e28395f265812603d50e307cb1b79a7c4”: failed to look up reserved IPs: connection is unauthorized: ipreservations.crd.projectcalico.org is forbidden: User “system:serviceaccount:kube-system:calico-node” cannot list resource “ipreservations” in API group “crd.projectcalico.org” at the cluster scope

It seems that happens after the rbac is enabled so how. Is that true? Any way to resolve it? Thanks!

This is odd, the ClusterRole for calico-node includes this crd.
Do you also see this error in your linux node?

Not in the Linux node; only in windows

Can you do a kubectl get daemonset -A
Do you see 2 different daemonsets? One for linux and one for windows?

Im just guessing here, the ClusterRole may not be available for the serviceaccount used for the calico in windows.

Due to the urgency, I recreate the whole cluster. But this error happened twice before. I’ll try to reproduce it and provide more information here. Thanks indeed!