# Backup a cluster with Velero

**URL:** https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034
**Category:** microk8s
**Tags:** docs
**Created:** [February 10, 2022, 11:51am UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034 "2022-02-10T11:51:22Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)
#### Post date: [February 10, 2022, 11:51am UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034/1 "2022-02-10T11:51:22Z")

</div>

[Velero](https://velero.io/) is a popular open source backup solution for Kubernetes. Its core implementation is a controller running in the cluster that oversees the backup and restore operations. The administrator is given a CLI tool to schedule operations and/or perform on-demand backup and restores. This CLI tool creates Kubernetes resources that the in-cluster Velero controller acts upon. During installation the controller needs to be [configured with a repository (called a ‘provider’)](https://velero.io/docs/v1.8/supported-providers/), where the backup files are stored.

This document describes how to setup Velero with the MinIO provider acting as an S3 compatible object store.

## Prerequisites

### Enabling required components

DNS and helm are needed for this setup:

```bash
sudo microk8s enable dns
sudo microk8s enable helm3

```

### Install MinIO

[MinIO](https://charts.min.io/) provides an S3 compatible interface over storage provisioned by Kubernetes. For the purposes of this guide, the `hostpath storage` add-on is used to satisfy the persistent volume claims:

```bash
sudo microk8s enable hostpath-storage

```

Helm is used to setup MinIO under the `velero` namespace:

```bash
sudo microk8s kubectl create namespace velero
sudo microk8s helm3 repo add minio https://helm.min.io
sudo microk8s helm3 install -n velero --set buckets[0].name=velero,buckets[0].policy=none,buckets[0].purge=false minio minio/minio

```

### Create a demo workload

The workload we will demonstrate the backup with is an NGINX deployment and a corresponding service under the `workloads` namespace. Create this setup with:

```bash
sudo microk8s kubectl create namespace workloads
sudo microk8s kubectl create deployment nginx -n workloads --image nginx
sudo microk8s.kubectl expose deployment nginx -n workloads --port 80

```

## Installing Velero

To install Velero we get the a binary from the [releases page on github](https://github.com/vmware-tanzu/velero/releases) and place it in our `PATH`. In this case we install the v1.7.1 Linux binary for AMD64 under `/usr/local/bin`:

```bash
wget https://github.com/vmware-tanzu/velero/releases/download/v1.7.1/velero-v1.7.1-linux-amd64.tar.gz 
tar -xzf velero-v1.7.1-linux-amd64.tar.gz
chmod +x velero-v1.7.1-linux-amd64/velero
sudo chown root:root velero-v1.7.1-linux-amd64/velero
sudo mv velero-v1.7.1-linux-amd64/velero /usr/local/bin/velero

```

Before installing Velero, we export the kubeconfig file from MicroK8s.

```bash
mkdir -p $HOME/.kube
sudo microk8s config > $HOME/.kube/config

```

We also export the MinIO credentials so we can feed them to Velero.

```bash
ACCESS_KEY=$(sudo microk8s kubectl -n velero get secret minio -o jsonpath="{.data.accesskey}" | base64 --decode)
SECRET_KEY=$(sudo microk8s kubectl -n velero get secret minio -o jsonpath="{.data.secretkey}" | base64 --decode)
cat <<EOF > credentials-velero
[default]
    aws_access_key_id=${ACCESS_KEY}
    aws_secret_access_key=${SECRET_KEY}
EOF

```

We are now ready to install Velero:

```bash
velero install \
--use-restic \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.3.0 \
--bucket velero \
--secret-file ./credentials-velero \
--backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://minio.velero.svc:9000 \
--snapshot-location-config region=minio

```

Velero uses [Restic](https://velero.io/docs/main/restic/.) for backing up Kubernetes volumes. To let Restic know of the kubelet directory in the MicroK8s context we need to patch its daemonset manifest:

```bash
sudo microk8s kubectl -n velero patch daemonset.apps/restic --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/volumes/0/hostPath/path", "value":"/var/snap/microk8s/common/var/lib/kubelet/pods"}]'

```

## Backup workloads

To backup the `workloads` namespace we use the `--include-namespaces` argument:

```bash
 velero backup create workloads-backup --include-namespaces=workloads

```

> ⓘ **Note:** Please, consult the [official Velero documentation](https://velero.io/docs/v1.10/file-system-backup/#to-back-up) on how to backup persistent volumes, the supported volume types and the limitations on hostpath.

To check the progress of a backup operation we use `describe`, providing the backup name:

```bash
 velero backup describe workloads-backup 

```

In the output you should see this operation completed:

```bash
Name: workloads-backup
Namespace: velero
Labels: velero.io/storage-location=default
Annotations: velero.io/source-cluster-k8s-gitversion=v1.23.3-2+3cea96839f0d64
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=23+

Phase: Completed

Errors: 0
Warnings: 0

Namespaces:
  Included: workloads
  Excluded: <none>

Resources:
  Included: *
  Excluded: <none>
  Cluster-scoped: auto

Label selector: <none>

Storage Location: default

Velero-Native Snapshot PVs: auto

TTL: 720h0m0s

Hooks: <none>

Backup Format Version: 1.1.0

Started: 2022-02-08 10:44:08 +0200 EET
Completed: 2022-02-08 10:44:10 +0200 EET

Expiration: 2022-03-10 10:44:08 +0200 EET

Total items to be backed up: 17
Items backed up: 17

Velero-Native Snapshots: <none included>

```

## Restore workloads

Before restoring the workloads namespace, let’s delete it first:

```bash
 sudo microk8s.kubectl delete namespace workloads

```

We can now create a restore operation specifying the backup we want to use:

```bash
velero restore create --from-backup workloads-backup

```

A restore operation which we can monitor using the describe command is then created:

```bash
velero restore describe workloads-backup-20220208105156

```

The `describe` output should eventually report a “Completed” phase:

```bash
Name: workloads-backup-20220208105156
Namespace: velero
Labels: <none>
Annotations: <none>

Phase: Completed
Total items to be restored: 10
Items restored: 10

Started: 2022-02-08 10:51:56 +0200 EET
Completed: 2022-02-08 10:51:57 +0200 EET

Backup: workloads-backup

Namespaces:
  Included: all namespaces found in the backup
  Excluded: <none>

Resources:
  Included: *
  Excluded: nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io
  Cluster-scoped: auto

Namespace mappings: <none>

Label selector: <none>

Restore PVs: auto

Preserve Service NodePorts: auto

```

Listing the resources of the `workloads` namespaces confirms that the restoration process was successful:

```bash
sudo microk8s kubectl get all -n workloads

```

## Summing up

Although Velero is a really powerful tool with a large set of configuration options it is also very easy to use. You are required to set up a backup strategy based on the backend that will hold the backups and the scheduling of the backups. The rest is taken care of by the tool itself.

---

<div class="post-metadata">

### Author: ![JellevdK](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/jellevdk/32/10108_2.png) [@JellevdK](https://discuss.kubernetes.io/u/JellevdK)
#### Post date: [May 19, 2022, 10:27am UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034/2 "2022-05-19T10:27:56Z")

</div>

I think one important option is missing in the velero install arguments: “–default-volumes-to-restic”. This enables the use of restic for taking a backup of all pod volumes as a default.  
More info: [Velero Docs - Use Tencent Cloud Object Storage as Velero's storage destination.](https://velero.io/docs/main/contributions/tencent-config/#install-velero-resources)

---

<div class="post-metadata">

### Author: ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)
#### Post date: [May 19, 2022, 12:24pm UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034/3 "2022-05-19T12:24:56Z")

</div>

@JellevdK thanks for that, I’ll take a look how we can add that to this page

---

<div class="post-metadata">

### Author: ![Nasir\_Mahmood](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/nasir_mahmood/32/11424_2.png) [@Nasir\_Mahmood](https://discuss.kubernetes.io/u/Nasir_Mahmood)
#### Post date: [December 20, 2022, 6:46pm UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034/4 "2022-12-20T18:46:59Z")

</div>

When the velero is configured with version 1.24+ channel using snap, the velero integration command returns

```auto
/usr/local/bin/velero install \
--secret-file=./credentials-velero \
--provider=aws \
--bucket=velero \
--backup-location-config region=minio-default,s3ForcePathStyle=true,s3Url=http://192.168.2.210:9000 region=minio-default \
--plugins=velero/velero-plugin-for-aws:v1.4.0 \
--use-volume-snapshots=true \
--use-restic=true \
--snapshot-location-config region=minio-default \
--wait

```

returns  
**An error occurred: unable to load root certificates: unable to parse bytes as PEM block**

The error **"An error occurred: unable to load root certificates: unable to parse bytes as PEM block**" happens only when we are using microk8s. however this error isn’t seen when using google kubernetes or k3s.

this happens becuase default config file is somehow stripped version of the actual kubeconfig file with below given contents

```auto
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://127.0.0.1:16443
  name: microk8s-cluster
contexts:
- context:
    cluster: microk8s-cluster
    user: admin
  name: microk8s
current-context: microk8s
kind: Config
preferences: {}
users:
- name: admin
  user:
    token: REDACTED

```

the  
`certificate-authority-data: DATA+OMITTED`  
and  
`token: REDACTED`  
dont let the velero work.

however, this approach works

Step 1 - microk8s config \> microk8s.config  
Step 2 - export KUBECONFIG=./microk8s.config  
Step 3 -

```auto
velero install --secret-file=./credentials-velero --provider=aws --bucket=velero --backup-location-config region=minio-default,s3ForcePathStyle=true,s3Url=http://192.168.2.210:9000 region=minio-default --plugins=velero/velero-plugin-for-aws:v1.4.0 --use-volume-snapshots=true --use-restic=true --snapshot-location-config region=minio-default --wait

CustomResourceDefinition/backups.velero.io: attempting to create resource
CustomResourceDefinition/backups.velero.io: attempting to create resource client
CustomResourceDefinition/backups.velero.io: created
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource client
CustomResourceDefinition/backupstoragelocations.velero.io: created
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource client
CustomResourceDefinition/deletebackuprequests.velero.io: created
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource client
CustomResourceDefinition/downloadrequests.velero.io: created
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource client
CustomResourceDefinition/podvolumebackups.velero.io: created
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource client
CustomResourceDefinition/podvolumerestores.velero.io: created
CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource
CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource client
CustomResourceDefinition/resticrepositories.velero.io: created
CustomResourceDefinition/restores.velero.io: attempting to create resource
CustomResourceDefinition/restores.velero.io: attempting to create resource client
CustomResourceDefinition/restores.velero.io: created
CustomResourceDefinition/schedules.velero.io: attempting to create resource
CustomResourceDefinition/schedules.velero.io: attempting to create resource client
CustomResourceDefinition/schedules.velero.io: created
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource client
CustomResourceDefinition/serverstatusrequests.velero.io: created
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource client
CustomResourceDefinition/volumesnapshotlocations.velero.io: created
Waiting for resources to be ready in cluster...
Namespace/velero: attempting to create resource
Namespace/velero: attempting to create resource client
Namespace/velero: already exists, proceeding
Namespace/velero: created
ClusterRoleBinding/velero: attempting to create resource
ClusterRoleBinding/velero: attempting to create resource client
ClusterRoleBinding/velero: created
ServiceAccount/velero: attempting to create resource
ServiceAccount/velero: attempting to create resource client
ServiceAccount/velero: created
Secret/cloud-credentials: attempting to create resource
Secret/cloud-credentials: attempting to create resource client
Secret/cloud-credentials: created
BackupStorageLocation/default: attempting to create resource
BackupStorageLocation/default: attempting to create resource client
BackupStorageLocation/default: created
VolumeSnapshotLocation/default: attempting to create resource
VolumeSnapshotLocation/default: attempting to create resource client
VolumeSnapshotLocation/default: created
Deployment/velero: attempting to create resource
Deployment/velero: attempting to create resource client
Deployment/velero: created
DaemonSet/restic: attempting to create resource
DaemonSet/restic: attempting to create resource client
DaemonSet/restic: created
Waiting for Velero deployment to be ready.
Waiting for Velero restic daemonset to be ready.
Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n velero' to view the status.

```

---

<div class="post-metadata">

### Author: ![timatgca](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/timatgca/32/13195_2.png) [@timatgca](https://discuss.kubernetes.io/u/timatgca)
#### Post date: [September 4, 2023, 5:53am UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034/5 "2023-09-04T05:53:18Z")

</div>

For velero v 1.11 it seems that the velero install command should use

`velero install --use-node-agent`

instead of the currently documented

```auto
velero install \
--use-restic \

```

and therefore the following step becomes

```auto

sudo microk8s kubectl -n velero patch daemonset.apps/node-agent --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/volumes/0/hostPath/path", "value":"/var/snap/microk8s/common/var/lib/kubelet/pods"}]'

```

---

<div class="post-metadata">

### Author: ![evilnick](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/evilnick/32/3481_2.png) [@evilnick](https://discuss.kubernetes.io/u/evilnick)
#### Post date: [September 8, 2023, 2:42pm UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034/6 "2023-09-08T14:42:04Z")

</div>

thanks! I will take a look and update

---

<div class="post-metadata">

### Author: ![Black-Byte](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/black-byte/32/14762_2.png) [@Black-Byte](https://discuss.kubernetes.io/u/Black-Byte)
#### Post date: [April 15, 2024, 12:19pm UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034/7 "2024-04-15T12:19:28Z")

</div>

Looks like the minio repo has changed to [https://charts.min.io/](https://charts.min.io/) .  
Source: [minio/helm/minio at master · minio/minio · GitHub](https://github.com/minio/minio/tree/master/helm/minio)

---

<div class="post-metadata">

### Author: ![suriya786](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/suriya786/32/17388_2.png) [@suriya786](https://discuss.kubernetes.io/u/suriya786)
#### Post date: [June 15, 2025, 2:51am UTC](https://discuss.kubernetes.io/t/backup-a-cluster-with-velero/19034/8 "2025-06-15T02:51:04Z")

</div>

Thank you so very much. Appreciate the time to put together a solution and shared the steps. regards and sincerely