Now docker has production ready multi-host networking capabilities and it has commands to manage networks from the command line. This guide walks you through the basic docker multi-host networking using docker machine and consul service discovery tool.
[alert style=”e.g. white, grey, red, yellow, green”] There is only one prerequisite for this tutorial. You just need the latest docker toolbox installed on your system.[/alert]
Docker Multi-Host Networking
In this setup, we will create three docker hosts on virtual box using docker machine. One host runs consul and other two hosts share the network information using consul service discovery container on the first host. You can learn more about consul from here.
7. Now, if you check the networks on host3, you will be able to see the overlay network we created on host2. It is because our two hosts are registered with consul and the network information is shared among all the hosts which are registered with it.
docker $(docker-machine config host2) network ls
docker $(docker-machine config host3) network ls
Now, if you launch containers in the differnt host, you will be able to connect them using the container name. Let test it by launching a Nginx container on host2 and test the connection by downloading the default Nginx page from host3 using a busybox container.
8. Launch a Nginx container on host2 by specifying the network “myapp” we have created.
docker $(docker-machine config host2) run -itd --name=webfront --net=myapp nginx
9. Verify the running container.
docker $(docker-machine config host2) ps
10. Now, launch a busybox container on host3 with parameters to download the homepage of nginx running on host2.
In part I, we learned the basic concepts of elasticsearch. In this tutorial, we will learn how to set up an elasticsearch cluster with client, master and a data node.
Setup an Elasticsearch Cluster
For this setup to work, as a prerequisite, you need three virtual machines with enough memory. This tutorial is based on ubuntu server 14.04. You can set up an ubuntu server using vagrant, or on any cloud provider.
Do the following before we start configuring the server for elasticsearch.
1. Create three ubuntu 14.04 VM’s with 1GB RAM each.
2. Update all the servers using the following command.
sudo apt-get update
3. Change the hostnames to es-client-01, es-master-01 and es-data-01 to match the client, master and data node roles.
4. Edit /etc/hosts file of all the nodes and make entries for all the nodes for the hostnames as shown below. Change the IP addresses with the IP addresses of your VM’s.
Note: At the time of writing, the release of elasticsearch is 2.2.0
2. Install the downloaded package.
Note: If you have downloaded any version other than 2.2.0, change the package name accordingly.
sudo dpkg -i elasticsearch-2.2.0.deb
3. Start the elasticsearch service.
sudo service elasticsearch start
4. Our node es-client-01 has elasticsearch service running and we will consider as the client node. Also, you need to set elasticsearch to start automatically on bootup. Use the following command to do that.
sudo update-rc.d elasticsearch defaults 95 10
5. Verify the elasticsearch service by sending a HTTP request to port 9200. By default elasticsearch run on port 9200.
You would see a JSON response, which looks like the following.
The above output shows the name of the node, cluster name, and a few other details.
If you do not specify a node name in the configuration, elasticsearch assigns a random name on every restart.
All the elasticsearch configurations are present in elasticsearch.yml file, which is located in /etc/elasticsearch folder.
6. Now, the elasticsearch.yml file has to be edited for the configuring the node as a client node. Open the elasticsearch.ym file located in /etc/elasticsearch directory and change the configurations as follows.
The configuration file has many sections like cluster, node, paths etc.
Note: Refer this config file for all the configurations explained below.
Under the cluster section, change the cluster name parameter.
Under node section, change the node name parameter and add other parameters as shown below.
The above parameters disable the multicast and send a unicast message to the specified hosts. As we have already made the hosts entry for all the hostnames, the unicast messages will go the respective nodes.
4. Save the file and restart the elasticsearch service for changes.
sudo service elasticsearch restart
5. Now, we need to make some system level changes. Open /etc/security/limits.conf file to change the file limits that can be used. By default, it is 1024 for Ubuntu. You can check this by running “unlimited -n” command.
Add the following lines at the end of the file.
* soft nofile 64000
* hard nofile 64000
root soft nofile 64000
root hard nofile 64000
6. Open /etc/pam.d/common-session file and add the following line.
session required pam_limits.so
7. Open /etc/pam.d/common-session-noninteractive and add the following.
session required pam_limits.so
8. It is recommended to have the heap size as half as the RAM. This tutorial is based on 1 GB RAM VM. So we will configure 512 MB swap space.
You need to set an environment variable for elasticsearch heap size. You can do this by editing the /etc/environment file. The file should look like the following.
Follow all the steps we used to setup the client node for the master and data node. Only while configuring the elasticsearch.yml file just uses the data given below. All the other steps are same for all the nodes.
For master node (elasticsearch.yml)
Under node section of the elasticsearch.yml file, add the following. Refer this file.