Backup and Restore ETCD Database

  • Are this the correct commands to backup the ETCD DB:

ETCDCTL_API=3 etcdctl help

ETCDCTL_API=3 etcdctl --endpoints https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key snapshot save /tmp/snapshot-pre-boot.db

  • Are this the correct commands to Restore the ETCD DB:

ETCDCTL_API=3 etcdctl --endpoints=https://[127.0.0.1]:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt
–name=master
–cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key
–data-dir /var/lib/etcd-from-backup
–initial-cluster=master=https://127.0.0.1:2380
–initial-cluster-token=etcd-cluster-1
–initial-advertise-peer-urls=https://127.0.0.1:2380
snapshot restore /tmp/snapshot-pre-boot.db

  • Update ETCD manifest file with correct backup directory and token:

  • –data-dir=/var/lib/etcd-from-backup

  • –initial-cluster-token=etcd-cluster-1

  • Update ETCD manifest file with correct hostpath and volume:
    volumeMounts:

    • mountPath: /var/lib/etcd-from-backup
      name: etcd-data
    • mountPath: /etc/kubernetes/pki/etcd
      name: etcd-certs

    volumes:

    • hostPath:
      path: /var/lib/etcd-from-backup
      type: DirectoryOrCreate
      name: etcd-data
    • hostPath:
      path: /etc/kubernetes/pki/etcd
      type: DirectoryOrCreate
4 Likes

You can visit this post. ETCD - backup and restore management

Above steps looks good, are you facing any issue?

First, you need to install etcd-client: sudo apt install etcd-client

can we restore etcd using pem format cert and keys? if so, is it possible only from master node?

ETCD backup is as below:
ETCDCTL_API=3 etcdctl --endpoints=https://[127.0.0.1]:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt
–name=master
–cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key snapshot save /var/etcd-backup.db

ETCD restore using below:

cert-/opt/etcd-client-ca.pem key=/opt/etcd-client-key.pem restore from /opt/previous_backup.db

Below steps worked for me, hope this helps:

Back up the etcd data:
ETCDCTL_API=3 etcdctl snapshot save /home/cloud_user/etcd_backup.db
–endpoints=https://etcd1:2379
–cacert=/home/cloud_user/etcd-certs/etcd-ca.pem
–cert=/home/cloud_user/etcd-certs/etcd-server.crt
–key=/home/cloud_user/etcd-certs/etcd-server.key

Restore the etcd Data from the Backup

1Stop etcd:
sudo systemctl stop etcd

  1. Delete the existing etcd data
    sudo rm -rf /var/lib/etcd

  2. Restore etcd data from a backup:
    sudo ETCDCTL_API=3 etcdctl snapshot restore /home/cloud_user/etcd_backup.db
    –initial-cluster etcd-restore=https://etcd1:2380
    –initial-advertise-peer-urls https://etcd1:2380
    –name etcd-restore
    –data-dir /var/lib/etcd

  3. Set database ownership:
    sudo chown -R etcd:etcd /var/lib/etcd

  4. Start etcd:
    sudo systemctl start etcd

Verify the system is working:
ETCDCTL_API=3 etcdctl get cluster.name
–endpoints=https://etcd1:2379
–cacert=/home/cloud_user/etcd-certs/etcd-ca.pem
–cert=/home/cloud_user/etcd-certs/etcd-server.crt
–key=/home/cloud_user/etcd-certs/etcd-server.keystrong text

also can run below command to check member list -

ETCDCTL_API=3 etcdctl --endpoints 10.2.0.9:2379 \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  member list

Know the ETCD version :

ETCDCTL_API=3 etcdctl version

Take a snapshot backup and save it to a different directory :

ETCDCTL_API=3 etcdctl --endpoints=https://[127.0.0.1]:2379 \

–cacert=/etc/kubernetes/pki/etcd/ca.crt \

–cert=/etc/kubernetes/pki/etcd/server.crt \

–key=/etc/kubernetes/pki/etcd/server.key \

snapshot save /opt/snapshot-pre-boot.db

Restore the ETCD backup

ETCDCTL_API=3 etcdctl --endpoints=https://[127.0.0.1]:2379 \

–cacert=/etc/kubernetes/pki/etcd/ca.crt \

–cert=/etc/kubernetes/pki/etcd/server.crt \

–key=/etc/kubernetes/pki/etcd/server.key \

–data-dir /var/lib/etcd-from-backup \

snapshot restore /opt/snapshot-pre-boot.db