Month: December 2019

Kubeadm is an excellent tool to set up a working kubernetes cluster in minutes. It does all the heavy lifting in terms of setting up all kubernetes components. It follows all the configuration best practices for a kubernetes cluster.

This blog post walks you through the process of setting up a kubernetes cluster with one master and two worker nodes using Kubeadm. I use kubeadm for all my kubernetes test clusters. You can set up the kubernetes cluster using kubeadm under 7 minutes.

Prerequisites:

1. Minimum two Ubuntu nodes [One master and one worker node]. You can have more worker nodes as per your requirement.
2. The master node should have a minimum for 2 vCPU and 6 GB memory.
3. 10.X.X.X/X network range for master and nodes. We will be using the 192 series as the pod network range. The Calico network plugin will use this range by default.

You Might Like: Get exclusive discount on kubernetes CKA/CKAD certifications

Port Requirements

Please refer to the following image and make sure all the ports are allowed for the control plane (master) and the worker nodes. If you set up this on a cloud, make sure you allow the ports in the firewall configuration.

On All The Nodes

Install Docker

As a first step, we need to install Docker on all the nodes. Execute the following commands on all the nodes.

Install the required packages for Docker.

sudo apt-get update && sudo apt-get install apt-transport-https ca-certificates curl software-properties-common -y

Add the Docker GPG key and apt repository.

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -

sudo add-apt-repository \
  "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) \
  stable"

Install the Docker community edition.

sudo apt-get update && apt-get install docker-ce=18.06.2~ce~3-0~ubuntu

Add the docker daemon configurations.

cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

Create a service directory.

mkdir -p /etc/systemd/system/docker.service.d

Restart Docker service.

systemctl daemon-reload
systemctl restart docker

Install Kubeadm & Kubelet & Kubectl

Install the required dependencies.

sudo apt-get update && sudo apt-get install -y apt-transport-https curl

Add the GPG key.

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

Add the kubernetes apt repository.

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

Update apt and install kubelet, kubeadm and kubectl.

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Now we have all the required utilities and tools for configuring Kubernetes components using kubeadm.

Initialize Kubeadm On Master Node

Initial kubeadm on master node with the following command. It will set up all the Kubernetes master components.

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

On a successful kubeadm initialization you should get the following output.

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.128.0.37:6443 --token j4eice.33vgvgyf5cxw4u8i \
    --discovery-token-ca-cert-hash sha256:37f94469b58bcc8f26a4aa44441fb17196a585b37288f85e22475b00c36f1c61

In the above output, there are two important blocks.

kubeconfig:

Use the following commands from the output to create the kubeconfig in master so that you can use kubectl to interact with cluster API.

Note: You can copy the admin.conf file from the master to your workstation if you don’t want to execute kubectl commands from the master.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Kubeadm Join Token:

The following command from the output is important to join the worker nodes to the master.

kubeadm join 10.128.0.37:6443 --token j4eice.33vgvgyf5cxw4u8i \
    --discovery-token-ca-cert-hash sha256:37f94469b58bcc8f26a4aa44441fb17196a585b37288f85e22475b00c36f1c61

Install Calico Network Plugin:

Execute the following command to install the calico network plugin on the cluster. Make sure you execute the kubectl command from where you have configured the kubeconfig file.

kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml

Check master node status using the following command.

kubectl get nodes

On Nodes

On all the nodes, execute the kubeadm join command you got from the output.

kubeadm join 10.128.0.37:6443 --token j4eice.33vgvgyf5cxw4u8i \
    --discovery-token-ca-cert-hash sha256:37f94469b58bcc8f26a4aa44441fb17196a585b37288f85e22475b00c36f1c61

From the master node, execute the following command to check if the node is added to the master.

kubectl get nodes

Output:

[email protected]:~# kubectl get nodes
NAME          STATUS   ROLES    AGE   VERSION
kube-master   Ready    master   10m   v1.16.3
node-01       Ready    <none>   32s   v1.16.3

Setup Kubernetes Metrics Server

Kubeadm doesn’t install metrics server components during its initialization. We have to install it separately.

Clone the metrics server repo from Github.

git clone https://github.com/kubernetes-sigs/metrics-server.git

Open the metrics server deployment file.

vi metrics-server/deploy/1.8+/metrics-server-deployment.yaml

Under container args, make sure you add the extra two parameters --kubelet-insecure-tls & --kubelet-preferred-address-types=InternalIP as shown below.

containers:
      - name: metrics-server
        image: k8s.gcr.io/metrics-server-amd64:v0.3.6
        args:
          - --cert-dir=/tmp
          - --secure-port=4443
          - --kubelet-insecure-tls
          - --kubelet-preferred-address-types=InternalIP

Deploy the metrics server objects using the following command.

kubectl apply -f metrics-server/deploy/1.8+/

Once the metrics server objects get deployed, it takes a minute for you to see the node and pod metrics using the top command.

kubectl top nodes

You should be able to view the node metrics as shown below.

[email protected]:/home/bibin.w# kubectl top nodes
NAME      CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
master    134m         6%     929Mi           12%
node-01   86m          8%     888Mi           24%
node-02   54m          5%     872Mi           24%

kubectl top po -n kube-system

Deploy A Sample Application

Create an Nginx deployment.

kubectl run nginx --image=nginx

You Might Like: Kubernetes Deployment Tutorial

Expose the Nginx deployment on a NodePort.

kubectl expose deployment nginx --type=NodePort --name=nginx-service

Ge the assigned NodePort using the following command.

kubectl describe svc nginx-service

You should be able to access Nginx on the allocated NodePort.

References:

1. Kubernetes Official Kubeadm Document