How to setup ELK stack on Amazon EC2 – Getting Started Guide

ELK stack on amazon ec2
ELK is the most popular log aggreration and analysis stack. ELK stands for Elasticsearch, Logstash and Kibana.

Elasticsearch – It is a NoSQL, analytics and search engine.

Logstash – It is a log aggregation tool.

Kibana – It is a dashboarding tool.

ELK stack on Amazon EC2

This guide will walk you through setting up a test ELK stack on amazon ec2 Ubuntu 14.04  instance. Follow the steps given below to have a working ELK stack.

Note: I assume that you have a ubuntu 14.04 server up and running in a public subnet.

1. Login and update the server.

2. ELK stack need openjdk. Install it using the following command.

3. Chef the jdk installion by checking its verison.

Installing Elasticsearch

4. Add the elasticsearch repository using the following command.

5. Elasticsearch key has to be added to the list using the following command.

6. Update the server

7. Install elasticsearch using the following command.

8. Execute the following command to start the elasticsearch server.

9. Test the elasticsearch service using curl

The output should look like the following.

READ  Title: How To Setup and Configure Latest Magento 2.x On Linux EC2

Installting and setting up Logstash

1. Add logstash to the list.

2. Update the server.

3. Install logstash.

4. Update the server.

5. Start and check the status of logstash using the following commands.

Since it is a single host instaltion, we will direct all our logs to logstach using a configuration file.

6. create a file named /etc/logstash/conf.d/logstash-syslog.conf and copy the following contents on to the file.

7. Now, restart the logstash server to make necessary changes.

READ  Elasticsearch Tutorial For Beginners - Getting Started Series

Installing and configuring kibana

1. Download the kibana source file.

2. Extract the source content

3. Create a kibana directory under /opt

4. Move the extracted folder contents to the /opt/kibana directory.

5. Create a service file for startup using the following commands.

6. Start the kibana dashboard service.

The above command would start kibana service on port 5601.

Now you can access the kibana dashboard on 5601 port with the public ip of your instance.

Free DevOps Resources

Get DevOps news, tutorials and resources in your inbox. A perfect way If you want to get started with devops. Like you, we dont like spam.

3 comments

  1. bjuggs Reply

    Great tutorial, thank you. I was unable to get sudo service kibana4 start to work. “kibana4: unrecognized service.” Any idea what might cause this? Thanks again!

  2. Vignesh Reply

    Great Tutorial. I installed ELK in a fresh AWS instance. And I followed the same steps that is provided above. Installed everything successfully. I started all the ELK services (i.e Elasticsearch, Logstash ,Kibana) too without any errors. But when I try to run the output in the browser, (http://localhost:5601 or http://IPAddress:5601 – Kibana UI) or (http://localhost:5601 or http://IPAddress:9200 – Elasticsearch), it says The output cannot be displayed. I am not sure if there is any error. Any help would be appreciated.

Leave a Reply

Your email address will not be published. Required fields are marked *