Backup of Kubernetes etcd database


#1

I want to take backup of kubernetes state. My etcd members list as as below:-

[root@kubemaster1 ~]# etcdctl member list
623b4dfff1a7e6f1: name=kubemaster3 peerURLs=http://10.235.67.4:2380 clientURLs=http://10.235.67.4:2379 isLeader=false
92afbbe7381285e0: name=kubemaster2 peerURLs=http://10.235.67.5:2380 clientURLs=http://10.235.67.5:2379 isLeader=false
99341b9cb2156c6e: name=kubemaster1 peerURLs=http://10.235.67.21:2380 clientURLs=http://10.235.67.21:2379 isLeader=true


#2

Gardener has an awesome way for Etcd backup and restore. Have a look https://github.com/gardener/etcd-backup-restore


#3

Thanks a ton for your response. Let me try that once.


#4

I also tried to configure Heptio Ark for backup. But only manges to install namespace & services


#5

This Etcd Backup & restore which we use internally for gardener sits as a sidecar to etcd and it takes full snapshot and even delta snapshots on time intervals and uploads to cloud storages which are also configured according to your own IAAS and whenever there is a or restart of etcd or some failures , it will automatically restores from the snapshot


#6

Sounds great ! I will try it tonight for sure


#7

I am not using GCP or AWS . I am using Flexible Engine Cloud Services So can we integrate this here well ?

Although I have installed gardener


#8

I am getting below error with taking the backup of etcd database:-

[root@ggnfekube ~]# etcdbrctl snapshot -e --etcd-endpoints http://10.235.67.4:2379
INFO[0000] Validating schedule…
INFO[0000] Defragmentation period :72 hours
INFO[0000] Taking scheduled snapshot for time: 2019-02-07 06:58:33.87787256 +0000 UTC
WARN[0030] Taking scheduled snapshot failed: failed to get etcd latest revision: context deadline exceeded
FATA[0030] Snapshotter failed with error: failed to get etcd latest revision: context deadline exceeded