How To Install and Configure Prometheus On a Linux Server

Install and configure prometheus on linux

Prometheus is an open-source monitoring system which is very lightweight and has a good alerting mechanism.

Install and Configure Prometheus

This guide explains how to install and configure the latest Prometheus on a Linux VM.

If you would like to install Prometheus on a Kubernetes cluster, please see the Prometheus on kubernetes guide.

Before You Begin

  1. Ensure that you have sudo access to the Linux server because the commands used in this guide require elevated privileges.
  2. The server has access to the internet for downloading the Prometheus binary.
  3. Most importantly, firewall rules opened for accessing Prometheus port 9090 on the server.

Setup Prometheus Binaries

Step 1: Update the yum package repositories.

sudo yum update -y

Step 2: Go to the official Prometheus downloads page and get the latest download link for the Linux binary.

prometheus linux download link

Step 3: Download the source using curl, untar it, and rename the extracted folder to prometheus-files.

curl -LO url -LO https://github.com/prometheus/prometheus/releases/download/v2.22.0/prometheus-2.22.0.linux-amd64.tar.gz
tar -xvf prometheus-2.22.0.linux-amd64.tar.gz
mv prometheus-2.22.0.linux-amd64 prometheus-files

Step 4: Create a Prometheus user, required directories, and make Prometheus the user as the owner of those directories.

sudo useradd --no-create-home --shell /bin/false prometheus
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
sudo chown prometheus:prometheus /etc/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus

Step 5: Copy prometheus and promtool binary from prometheus-files folder to /usr/local/bin and change the ownership to prometheus user.

sudo cp prometheus-files/prometheus /usr/local/bin/
sudo cp prometheus-files/promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool

Step 6: Move the consoles and console_libraries directories from prometheus-files to /etc/prometheus folder and change the ownership to prometheus user.

sudo cp -r prometheus-files/consoles /etc/prometheus
sudo cp -r prometheus-files/console_libraries /etc/prometheus
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries

Setup Prometheus Configuration

All the prometheus configurations should be present in /etc/prometheus/prometheus.yml file.

Step 1: Create the prometheus.yml file.

sudo vi /etc/prometheus/prometheus.yml

Step 2: Copy the following contents to the prometheus.yml file.

global:
  scrape_interval: 10s

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']

Step 3: Change the ownership of the file to prometheus user.

sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

Setup Prometheus Service File

Step 1: Create a prometheus service file.

sudo vi /etc/systemd/system/prometheus.service

Step 2: Copy the following content to the file.

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

Step 3: Reload the systemd service to register the prometheus service and start the prometheus service.

sudo systemctl daemon-reload
sudo systemctl start prometheus

Check the prometheus service status using the following command.

sudo systemctl status prometheus

The status should show the active state as shown below.

prometheus status

Access Prometheus Web UI

Now you will be able to access the prometheus UI on 9090 port of the prometheus server.

http://<prometheus-ip>:9090/graph

You should be able to see the following UI as shown below.

prometheus web UI

You can use the prometheus query tab to query the available metrics as shown in the gig below.

querying metrics from prometheus UI

Right now, we have just configured the Prometheus server. You need to register the target in the prometheus.yml file to get the metrics from the source systems.

For example, if you want to monitor ten servers, the IP address of these servers should be added as a target in the Prometheus configuration to scrape the metrics.

The server should have Node Exporter installed to collect all the system metrics and make it available for Prometheus to scrap it.

Follow this detailed Prometheus Node Exporter Guide to setup node exporter and registering it to the Prometheus server. 

You can also use this setup as lab for the Prometheus Certified Associate Certification preparation.

9 comments
  1. There is some issue if you install node exporter on multiple servers and add configuration in Prometheus.yml scrape it crash Prometheus’s it does not start.
    We have to run node exporters on unique ports?

    1. HI Libra,

      Prometheus is capable of handling many targets. When you say Prometheus is crashing, are you seeing some errors in the logs? also, what is the system configuration that you are using?

      1. Running 2.13.1 prometheus version
        I found the issue and maybe its syntax errror?
        if I use all nodeexporter ip in one job_name line it works like this

        – targets: [‘localhost:9090′,’xxxxxx:9100′,’yyyyyyyyy:9100’]

        But when I define individual job names like this

        – job_name: ‘xxxxxxxx’
        static_configs:
        – targets: [‘xxxxxxxxx:9100’]

        – job_name: ‘yyyyyyyy’
        static_configs:
        – targets: [‘yyyyyyyyy:9100’]

        it crash

        server.com systemd[1]: Stopped Prometheus Server.
        server.com systemd[1]: start request repeated too quickly for prometheus.service
        server.com systemd[1]: Failed to start Prometheus Server.
        server.com systemd[1]: Unit prometheus.service entered failed state.
        server.com systemd[1]: prometheus.service failed.

        1. You can try in the following format as well

          static_configs:
              - targets: ['x.x.x.x:9100']
              - targets: ['x.x.x.x:9100']
              - targets: ['x.x.x.x:9100']
          
  2. Can i get the VM requirements for installing prometheus on an linux machine like No. of CPU, Storage space etc

Leave a Reply

Your email address will not be published. Required fields are marked *

You May Also Like