How To Setup an etcd Cluster On Linux – Beginners Guide

This blog will teach you the basics of etcd and then set up an etcd cluster on Linux using step-by-step guides

Before you begin, make sure you have the following setup.

1. Three Linux servers (Can be an odd quorum of 5, 7, etc based on the needs)
2. A valid hostname for all the servers
3. Firewall rules enabled all the servers on the following ports for client requests and peer-to-peer communication.

   2380 2379

Setup etcd Cluster

etcd setup is fairly easy and this guide follows the static bootstrap method, which means you need to know the IPs of your nodes for bootstrapping. This guide covers all the necessary steps to set up a cluster on Linux servers. It is a multimode setup with systemd files to run etcd as a service.

Following are the etcd server hostname and IP details used in this guide. Change the IPs mentioned in the guide with your IPs where ever needed.

1. etcd-1: 10.128.0.2
2. etcd-2: 10.128.0.4
3. etcd-3: 10.128.0.3

Let’s get started with the setup.

On All the 3 Nodes

Perform steps 1 to 6 on all three nodes.

Step 1: CD into the local src folder

cd /usr/local/src

Step 2: Download the latest etcd release from the etcd Github Releases. At the time of writing this article, the latest version is 3.3.10

sudo wget "https://github.com/coreos/etcd/releases/download/v3.3.9/etcd-v3.3.9-linux-amd64.tar.gz"

Step 3: Untar the binary.

sudo tar -xvf etcd-v3.3.9-linux-amd64.tar.gz

Step 4: Move the extracted etcd executables (etcd & ectdctl) to local bin.

sudo mv etcd-v3.3.9-linux-amd64/etcd* /usr/local/bin/

Step 5: Create relevant etcd folders, user & group. We will be running the etcd service as an etcd user.

sudo mkdir -p /etc/etcd /var/lib/etcd
groupadd -f -g 1501 etcd
useradd -c "etcd user" -d /var/lib/etcd -s /bin/false -g etcd -u 1501 etcd
chown -R etcd:etcd /var/lib/etcd

Step 6: Perform the following as root user.

Set two environment variables. One to fetch the system IP and another to get the system hostname.

ETCD_HOST_IP=$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)
ETCD_NAME=$(hostname -s)

Create a systemd service file for etcd. Replace --listen-client-urls with your server IPs

Note: –name , –initial-advertise-peer-urls, –listen-peer-urls, –listen-client-urls will be different for each server. ETCD_NAME & ETCD_HOST_IP variables will automatically replace it.

cat << EOF > /lib/systemd/system/etcd.service
[Unit]
Description=etcd service
Documentation=https://github.com/coreos/etcd
 
[Service]
User=etcd
Type=notify
ExecStart=/usr/local/bin/etcd \\
 --name ${ETCD_NAME} \\
 --data-dir /var/lib/etcd \\
 --initial-advertise-peer-urls http://${ETCD_HOST_IP}:2380 \\
 --listen-peer-urls http://${ETCD_HOST_IP}:2380 \\
 --listen-client-urls http://${ETCD_HOST_IP}:2379,http://127.0.0.1:2379 \\
 --advertise-client-urls http://${ETCD_HOST_IP}:2379 \\
 --initial-cluster-token etcd-cluster-1 \\
 --initial-cluster etcd-1=http://10.142.0.2:2380,etcd-2=http://10.142.0.4:2380,etcd-3=http://10.142.0.3:2380 \\
 --initial-cluster-state new \\
 --heartbeat-interval 1000 \\
 --election-timeout 5000
Restart=on-failure
RestartSec=5
 
[Install]
WantedBy=multi-user.target
EOF

Bootstrap The etcd Cluster

Once all the configurations are applied on the three servers, start and enable the newly created etcd service on all the nodes. The first server will act as a bootstrap node. One node will be automatically elected as a leader once the service is started in all the three nodes.

systemctl daemon-reload
systemctl enable etcd
systemctl start etcd.service
systemctl status -l etcd.service

Verify etcd Cluster Status

ectdctl is the utility to interact with the etcd cluster. You can find this utility in the folder/usr/local/bin of all the nodes.

You can use any one of the cluster nodes to perform the following checks.

Check the cluster health using the following command

etcdctl cluster-health

Verify cluster membership status using the following command. It will show the leader’s status.

etcdctl  member list

By default, etcdctl uses etcd v2. So you need to explicitly use a variable ETCDCTL_API=3 to access etcd v3 functionalities.

You can set it as an environment variable or pass it along with each etcdctl command as shown below.

Let’s write a few key-value pairs in the cluster and verify it.

ETCDCTL_API=3 etcdctl put name1 batman
ETCDCTL_API=3 etcdctl put name2 ironman
ETCDCTL_API=3 etcdctl put name3 superman
ETCDCTL_API=3 etcdctl put name4 spiderman

Now you can try getting the value of name3 using the following command.

ETCDCTL_API=3 etcdctl get name3

You can list all the keys using ranges and prefixes

ETCDCTL_API=3 etcdctl get name1 name4 # lists range name1 to name 4
ETCDCTL_API=3 etcdctl get --prefix name # lists all keys with name prefix