How to Backup etcd and Restore it on Kubernetes Cluster

backup etcd and restore it on Kubernetes

In this kubernetes tutorial, you will learn to backup etcd key-value store and restore it back on Kubernetes with an etcd snapshot.

In Kubernetes architecture, etcd is an integral part of the cluster. All the cluster objects and their state is stored in etcd. Few things you should know about etcd from a Kubernetes perspective.

  1. It is a consistent, distributed, and a secure key-value store.
  2. It uses raft protocol.
  3. Supports highly available architecture with stacked etcd.
  4. It stores kubernetes cluster configurations, all API objects, object states, and service discovery details.

Also, if you consider Kubernetes design best practices, Kubernetes etcd backup and restore is one of the important aspects under the backup strategy.

Kubernetes etcd Backup Using etcdctl

Here is what you should know about etcd backup.

  1. etcd has a built-in snapshot mechanism.
  2. etcdctl is the command line utility that interacts with etcd for snapshots.

Follow the steps given below to take an etcd snapshot.

Step 1: Log in to the control plane.

Step 2: If you don’t have etcdctl in your cluster control plane, install it using the following command.

sudo apt install etcd-client

Step 3: We need to pass the following three pieces of information to etcdctl to take an etcd snapshot.

  1. etcd endpoint (–endpoints)
  2. ca certificate (–cacert)
  3. server certificate (–cert)
  4. server key (–key)

You can get the above details in two ways.

From the etcd static pod manifest file located at /etc/kubernetes/manifests/etcd.yaml the location.

getting cert files from etcd static pod manifest

You can also get the above details by describing the etcd pod running in the kube-system namespace. While describing the pod, replace etcd-master-node with your etcd pod name.

kubectl get po -n kube-system
kubectl describe pod etcd-master-node -n kube-system

Step 4: Take an etcd snapshot backup using the following command.

ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=<ca-file> \
  --cert=<cert-file> \
  --key=<key-file> \
  snapshot save <backup-file-location>

The command looks like the following when you add the actual location and parameters. Execute the command to perform a backup. You can replace /opt/backup/etcd.db with the location and name of your choice.

ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  snapshot save /opt/backup/etcd.db

On a successful execution you will get a “Snapshot saved at /opt/backup/etcd.db”, message as shown below.

etcd backup snapshot command

Also, you can verify the snapshot using the following command.

ETCDCTL_API=3 etcdctl --write-out=table snapshot status /opt/backup/etcd.db

Here is an example output.

+----------+----------+------------+------------+
|   HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |
+----------+----------+------------+------------+
| b7147656 |    51465 |       1099 |     5.1 MB |
+----------+----------+------------+------------+

Kubernetes etcd Restore Using Snapshot Backup

Now we have the backup in the /opt/backup/etcd.db location. We will use the snapshot backup to restore etcd.

Here is the command to restore etcd.

ETCDCTL_API=3 etcdctl snapshot restore <backup-file-location>

Let’s execute the etcd restore command. In my case /opt/backup/etcd.db is the backup file.

ETCDCTL_API=3 etcdctl snapshot restore /opt/backup/etcd.db

If you want to use a specific data directory for the restore, you can add the location using the --data-dir flag as shown below.

ETCDCTL_API=3 etcdctl --data-dir /opt/etcd snapshot restore /opt/backup/etcd.db

Conclusion

In this blog, we learned Kubernetes etcd backup and restore using etcdctl command line utility.

etcd backup, and restore are essential tasks in Kubernetes cluster administration. Also, it is an important topic in the CKA certification exam.

If you are preparing for the CKA exam, do check out the CKA exam study guide and for the exam discount voucher, check the Kubernetes certification coupon.

1 Shares:
Leave a Reply

Your email address will not be published. Required fields are marked *

You May Also Like