Etcd Backup and Restore on Kubernetes Cluster [Tutorial]

In this kubernetes tutorial, you will learn the etcd backup and restore on Kubernetes cluster with an etcd snapshot.

In Kubernetes architecture, etcd is an integral part of the cluster. All the cluster objects and their state is stored in etcd. Few things you should know about etcd from a Kubernetes perspective.

1. It is a consistent, distributed, and secure key-value store.
2. It uses raft protocol.
3. Supports highly available architecture with stacked etcd.
4. It stores kubernetes cluster configurations, all API objects, object states, and service discovery details.

If you want to understand more about etcd and how kubernetes uses it, I recommend reading the comprehensive Kubernetes Architecture post.

Also, if you consider Kubernetes design best practices, Kubernetes etcd backup and restore is one of the important aspects under the backup strategy.

Kubernetes etcd Backup Using etcdctl

Here is what you should know about etcd backup.

1. etcd has a built-in snapshot mechanism.
2. etcdctl is the command line utility that interacts with etcd for snapshots.

Follow the steps given below to take an etcd snapshot.

Step 1: Log in to the control plane.

Step 2: If you don’t have etcdctl in your cluster control plane, install it using the following command.

sudo apt install etcd-client

Step 3: We need to pass the following three pieces of information to etcdctl to take an etcd snapshot.

1. etcd endpoint (–endpoints)
2. ca certificate (–cacert)
3. server certificate (–cert)
4. server key (–key)

You can get the above details in two ways.

From the etcd static pod manifest file located at /etc/kubernetes/manifests/etcd.yaml the location.

You can also get the above details by describing the etcd pod running in the kube-system namespace. While describing the pod, replace etcd-master-node with your etcd pod name.

kubectl get po -n kube-system
kubectl describe pod etcd-master-node -n kube-system

Step 4: Take an etcd snapshot backup using the following command.

ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=<ca-file> \
  --cert=<cert-file> \
  --key=<key-file> \
  snapshot save <backup-file-location>

The command looks like the following when you add the actual location and parameters. Execute the command to perform a backup. You can replace /opt/backup/etcd.db with the location and name of your choice.

ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  snapshot save /opt/backup/etcd.db

On a successful execution you will get a “Snapshot saved at /opt/backup/etcd.db”, message as shown below.

Also, you can verify the snapshot using the following command.

ETCDCTL_API=3 etcdctl --write-out=table snapshot status /opt/backup/etcd.db

Here is an example output.

+----------+----------+------------+------------+
|   HASH   | REVISION | TOTAL KEYS | TOTAL SIZE |
+----------+----------+------------+------------+
| b7147656 |    51465 |       1099 |     5.1 MB |
+----------+----------+------------+------------+

Kubernetes etcd Restore Using Snapshot Backup

Now we have the backup in the /opt/backup/etcd.db location. We will use the snapshot backup to restore etcd.

Here is the command to restore etcd.

ETCDCTL_API=3 etcdctl snapshot restore <backup-file-location>

Let’s execute the etcd restore command. In my case /opt/backup/etcd.db is the backup file.

ETCDCTL_API=3 etcdctl snapshot restore /opt/backup/etcd.db

If you want to use a specific data directory for the restore, you can add the location using the --data-dir flag as shown below.

ETCDCTL_API=3 etcdctl --data-dir /opt/etcd snapshot restore /opt/backup/etcd.db

etcd Backup FAQs

Conclusion

In this blog, we learned Kubernetes etcd backup and restore using etcdctl command line utility.

etcd backup, and restore are essential tasks in Kubernetes cluster administration. Also, it is an important topic in the CKA certification exam.

If you are preparing for the CKA exam, do check out the CKA exam study guide and for the exam discount voucher, check the Kubernetes certification coupon.