Now docker has production ready multi-host networking capabilities and it has commands to manage networks from the command line. This guide walks you through the basic docker multi-host networking using docker machine and consul service discovery tool.
[alert style=”e.g. white, grey, red, yellow, green”] There is only one prerequisite for this tutorial. You just need the latest docker toolbox installed on your system.[/alert]
Docker Multi-Host Networking
In this setup, we will create three docker hosts on virtual box using docker machine. One host runs consul and other two hosts share the network information using consul service discovery container on the first host. You can learn more about consul from here.
[alert style=”e.g. white, grey, red, yellow, green”] To know more about networking basics, refer this official docker documentation[/alert]
Follow the steps given below to set up multi-host networking.
1. Create a docker machine named “host1-consul”
docker-machine create -d virtualbox host1-consul
2. Launch a consul container on the host1-consul host using the following docker run command.
7. Now, if you check the networks on host3, you will be able to see the overlay network we created on host2. It is because our two hosts are registered with consul and the network information is shared among all the hosts which are registered with it.
docker $(docker-machine config host2) network ls
docker $(docker-machine config host3) network ls
Now, if you launch containers in the differnt host, you will be able to connect them using the container name. Let test it by launching a Nginx container on host2 and test the connection by downloading the default Nginx page from host3 using a busybox container.
8. Launch a Nginx container on host2 by specifying the network “myapp” we have created.
docker $(docker-machine config host2) run -itd --name=webfront --net=myapp nginx
9. Verify the running container.
docker $(docker-machine config host2) ps
10. Now, launch a busybox container on host3 with parameters to download the homepage of nginx running on host2.
In this tutorial, I will explain how to set up an NFS server and client configurations for mounting an NFS share across systems in a network.[alert style=”e.g. white,
[alert style=”e.g. white, grey, red, yellow, green”] Note: This tutorial is based on Ubuntu 14.04 server. You can create a VM easily on the cloud or using vagrant. [/alert]
Setup an NFS server
In this setup, I have a ubuntu server with IP address 192.168.0.2.
1. Refresh the apt package index
sudo apt-get update
2. Install the NFS package.
sudo apt-get install nfs-kernel-server
3. Next, we need to create a directory that can be shared with other hosts in the network. I am going to create a folder in var directory.
sudo mkdir /var/nfs
4. Change the ownership of the NFS folder to “nobody” and “nogroup”.
sudo chown nobody:nogroup /var/nfs
The “nobody” is a user present in most of the Linux distros which belong to the “nogroup” which does not have any privileges on the system programs or files.
5. All the NFS configurations are set in the /etc/exports file. In which we can give specific permissions for a client to access the files in the share.
In this example, I have a client with IP 192.168.0.3. Open the exports file and make an entry as shown below.
/var/nfs 192.168.0.3(rw,sync,no_subtree_check)
6. NFS table holds all the exports of the shares. You can create one using the following command.
sudo exportfs -a
7. Now, let’s start the NFS service.
sudo service nfs-kernel-server start
That’s it! We have an NFS server up and running.
Setup NFS client Node
All the servers which need access to the NFS share need to install the NFS client packages.
1. Refresh the apt list and install the client package.
In part I, we learned the basic concepts of elasticsearch. In this tutorial, we will learn how to set up an elasticsearch cluster with client, master and a data node.
Setup an Elasticsearch Cluster
For this setup to work, as a prerequisite, you need three virtual machines with enough memory. This tutorial is based on ubuntu server 14.04. You can set up an ubuntu server using vagrant, or on any cloud provider.
Do the following before we start configuring the server for elasticsearch.
1. Create three ubuntu 14.04 VM’s with 1GB RAM each.
2. Update all the servers using the following command.
sudo apt-get update
3. Change the hostnames to es-client-01, es-master-01 and es-data-01 to match the client, master and data node roles.
4. Edit /etc/hosts file of all the nodes and make entries for all the nodes for the hostnames as shown below. Change the IP addresses with the IP addresses of your VM’s.
Note: At the time of writing, the release of elasticsearch is 2.2.0
2. Install the downloaded package.
Note: If you have downloaded any version other than 2.2.0, change the package name accordingly.
sudo dpkg -i elasticsearch-2.2.0.deb
3. Start the elasticsearch service.
sudo service elasticsearch start
4. Our node es-client-01 has elasticsearch service running and we will consider as the client node. Also, you need to set elasticsearch to start automatically on bootup. Use the following command to do that.
sudo update-rc.d elasticsearch defaults 95 10
5. Verify the elasticsearch service by sending a HTTP request to port 9200. By default elasticsearch run on port 9200.
curl http://localhost:9200
You would see a JSON response, which looks like the following.
The above output shows the name of the node, cluster name, and a few other details.
If you do not specify a node name in the configuration, elasticsearch assigns a random name on every restart.
All the elasticsearch configurations are present in elasticsearch.yml file, which is located in /etc/elasticsearch folder.
6. Now, the elasticsearch.yml file has to be edited for the configuring the node as a client node. Open the elasticsearch.ym file located in /etc/elasticsearch directory and change the configurations as follows.
The configuration file has many sections like cluster, node, paths etc.
Note: Refer this config file for all the configurations explained below.
Under the cluster section, change the cluster name parameter.
cluster.name: devopscube-production
Under node section, change the node name parameter and add other parameters as shown below.
The above parameters disable the multicast and send a unicast message to the specified hosts. As we have already made the hosts entry for all the hostnames, the unicast messages will go the respective nodes.
4. Save the file and restart the elasticsearch service for changes.
sudo service elasticsearch restart
5. Now, we need to make some system level changes. Open /etc/security/limits.conf file to change the file limits that can be used. By default, it is 1024 for Ubuntu. You can check this by running “unlimited -n” command.
Add the following lines at the end of the file.
* soft nofile 64000
* hard nofile 64000
root soft nofile 64000
root hard nofile 64000
6. Open /etc/pam.d/common-session file and add the following line.
session required pam_limits.so
7. Open /etc/pam.d/common-session-noninteractive and add the following.
session required pam_limits.so
8. It is recommended to have the heap size as half as the RAM. This tutorial is based on 1 GB RAM VM. So we will configure 512 MB swap space.
You need to set an environment variable for elasticsearch heap size. You can do this by editing the /etc/environment file. The file should look like the following.
Follow all the steps we used to setup the client node for the master and data node. Only while configuring the elasticsearch.yml file just uses the data given below. All the other steps are same for all the nodes.
For master node (elasticsearch.yml)
Under node section of the elasticsearch.yml file, add the following. Refer this file.
Under the network section, replace the data nodes IP address as you did for the client and master nodes.
Once you configure all the three nodes, restart the elasticsearch service on all the three nodes.
sudo service elasticsearch restart
Now you will have a working elasticsearch cluster.
Installing elasticsearch GUI plugin
Once you setup an elasticsearch cluster, you can view the cluster status on the client node(es-client-01) using the following command.
curl http://es-client-01:9200/_cluster/stats
But the but is not that easy to comprehend. So you can make use of the elasticsearch head plugin to view the cluster details in the browser UI.
We will install this plugin on our client node. To install the plugin, navigate to “/usr/share/elasticsearch/bin” directory and execute the following command.
./plugin install mobz/elasticsearch-head
Restart the elasticsearch service for the plugin to work.
sudo service elasticsearch restart
Now, if you access http://<IP>:9200/_plugin/head/ in your browser, you will be able to see all the cluster details.
Wrapping Up
In this tutorial, I have explained all the steps to setup a three-node elasticsearch cluster. In the next article, I will cover more on indexing strategies for elasticsearch.
