How To Install and Configure Prometheus On a Linux Server

Install and configure prometheus on linux

Prometheus is an open source monitoring system which is very lightweight and has a good alerting mechanism.

Install and Configure Prometheus

This guide explains how to install and configure the latest Prometheus on a Linux VM.

If you would like to install Prometheus on a Kubernetes cluster, please see the Prometheus on kubernetes guide.

Before You Begin

  1. Ensure that you have sudo access to the Linux server because the commands used in this guide require elevated privileges.
  2. The server has access to the internet for downloading Prometheus binary.
  3. Most importantly, firewall rules opened for accessing Prometheus port 9090 on the server.

Setup Prometheus Binaries

Step 1: Update the yum package repositories.

sudo yum update -y

Step 2: Go to official Prometheus downloads page and get the download link for Linux binary.

prometheus linux download link

Step 3: Create a Prometheus user, required directories, and make prometheus user as the owner of those directories.

sudo useradd --no-create-home --shell /bin/false prometheus
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
sudo chown prometheus:prometheus /etc/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus

Step 4: Download the source using curl, untar it and rename the extracted folder to prometheus-files.

curl -LO https://github.com/prometheus/prometheus/releases/download/v2.3.2/prometheus-2.3.2.linux-amd64.tar.gz
tar -xvf prometheus-2.3.2.linux-amd64.tar.gz
mv prometheus-2.3.2.linux-amd64 prometheus-files

Step 5: Copy prometheus and promtool binary from prometheus-files folder to /usr/local/bin and change the ownership to prometheus user.

sudo cp prometheus-files/prometheus /usr/local/bin/
sudo cp prometheus-files/promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool

Step 6: Move the consoles and console_libraries directories from prometheus-files to /etc/prometheus folder and change the ownership to prometheus user.

sudo cp -r prometheus-files/consoles /etc/prometheus
sudo cp -r prometheus-files/console_libraries /etc/prometheus
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries

Setup Prometheus Configuration

All the prometheus configurations should be present in /etc/prometheus/prometheus.yml file.

Step 1: Create the prometheus.yml file.

sudo vi /etc/prometheus/prometheus.yml

Step 2: Copy the following contents to the prometheus.yml file.

global:
  scrape_interval: 10s

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']

Step 3: Change the ownership of the file to prometheus user.

sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

Setup Prometheus Service File

Step 1: Create a prometheus service file.

sudo vi /etc/systemd/system/prometheus.service

Step 2: Copy the following content to the file.

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

Step 3: Reload the systemd service to register the prometheus service and start the prometheus service.

sudo systemctl daemon-reload
sudo systemctl start prometheus

Check the prometheus service status using the following command.

sudo systemctl status prometheus

The status should show the active state as shown below.

prometheus status

Access Prometheus Web UI

Now you will be able to access the prometheus UI on 9090 port of the prometheus server.

http://<prometheus-ip>:9090/graph

You should be able to see the following UI as shown below.

prometheus web UI

Right now we have just configured prometheus. You need to register the target in the prometheus.yml file to get the metrics from the source systems.

Follow Prometheus Node Exporter Guide to setup node exporter and registering it to Prometheus server.


Install and configure prometheus on linux

ONLINE COURSE: Monitoring and Alerting with Prometheus

Monitoring and Alerting of your infra and apps using the leading open-source monitoring system. Visualize using Grafana

  • Learn to setup and configure prometheus
  • Learn to setup alerting on prometheus
  • Configure grafana for visualizing the monitoring data.
  • Learn to instrument application for prometheus monitoring.
  • Understand monitoring and alerting setup for your whole infrastructure.

 

Install and configure prometheus on linux

Setup NFS Server On Google Cloud – Managed Cloud FileStore Service

google cloud filestore - NFS setup on cloud

Google Cloud Filestore is a managed NFS implementation on google cloud. This is one of the awaited features for Google cloud users. Amazon AWS has released its Managed NFS service EFS in June 2016 with 99.99% availability.

Google Filestore Features

Key features of cloud filestore are shown in the table below. You can find the filestore pricing details here

cloud filestore key features

Google FileStore Setup

In this guide, I will walk you through the process of setting up a filestore using CLI and google cloud console.

Following are the topics covered.

  1. Creating filestore using CLI
  2. Creating filestore using Console
  3. Connecting to filestore from a Google Compute Instance

Create Filestore Using Gcloud CLI

Note: Filestore is in beta stage now

Step 1: Use the following syntax to create a filestore. Make sure you have Gcloud CLI installed

gcloud beta filestore instances create <name-of-filestore-instance> \
    --location=us-central1-c \
    --tier=STANDARD \
    --file-share=name="<filestore-name>",capacity=1TB \
    --network=name="default",reserved-ip-range="<IP-range>"

For example,

gcloud beta filestore instances create devopscube-nfs-server \
    --location=us-central1-c \
    --tier=STANDARD \
    --file-share=name="devopscubefileserver",capacity=1TB \
    --network=name="default",reserved-ip-range="10.0.0.0/29"

Step 2: You can list the available filestore instances using the following command.

gcloud beta filestore instances list

Step 3: You can describe a filestore using the following command.

gcloud beta filestore instances describe <filestore-instance-name> --location <region>

For example,

gcloud beta filestore instances describe devopscube-demo-nfs --location us-central1-a

You will get the following output.

createTime: '2018-08-16T18:26:46.864763Z'
fileShares:
- capacityGb: '1024'
  name: devopscubenfs
name: projects/devopscube/locations/us-central1-a/instances/devopscube-demo-nfs
networks:
- ipAddresses:
  - 10.179.213.42
  network: default
  reservedIpRange: 10.179.213.40/29
state: READY
tier: STANDARD

Create Filestore NFS Server From Cloud Console

Step1: Go to https://console.cloud.google.com/filestore/

Step 2: Click “Create Instance” option.

create NFS on google cloud

Step 3: Fill out the basic details as shown below.

create filestore option 1

Also, you can mention a custom network range in a selected network as shown below.

filestore address range

Step 4: The minimum size of NFS that can be created in 1 TB. Enter filestore name, required storage and click create. It will take a few minutes for the NFS instance to be created.

google filestore properties

Step 5: Click on the created instance to get the NFS details.

filestore details

It will show all the details such as mount point and networking address range as shown below.

Google filestore mount path

Connecting to filestore from a Google Compute Instance

Note: The instance which needs access to filestore storage should have access to the filestore network.

Step 1: Update the package list and install the NFS client.

sudo yum install nfs-utils

Step 2: Create a mount directory for mounting the filestore volume.

sudo mkdir /mnt/nfs-mount

Step 3: Mount the filestore volume to the mount point. You can get the mount point from the filestore details page as shown in previous steps.

sudo mount <filestore-IP>:/<filestore-name> /mnt/nfs-mount

For example,

sudo mount 10.179.213.42:/devopscubenfs /mnt/nfs-mount

Step 4: Check the mount point using df command.

df -h

You should see the mounted NFS as shown below.

mounting google filestore volume

Let us know in the comment section if you face any errors.

google cloud filestore - NFS setup on cloud

WordPress Installation Guide: Install and Configure Latest WordPress on Ubuntu Linux

wordpress installation guide

This WordPress installation guide helps you to install and configure the latest WordPress on an Ubuntu Linux box. It covers Apache, MariaDB, PHP and WordPress configurations.

WordPress Installation Guide

This installation guide covers the following.

  1. Installation and configuration of Apache2
  2. Installation and configuration for MariaDB
  3. Installation and configuration of PHP and Apache modules.
  4. WordPress apache configurations with a domain name.
  5. Accessing the WordPress dashboard.

Follow the steps given below for the complete WordPress setup on Ubuntu 16.04

Install and Configure Apache 2

Step1: Login to the server and update the package repos.

sudo apt-get update -y

Step 2: Install apache2

sudo apt-get install apache2 -y

Step 3: Execute the following command to disable anonymous directory listing.

sudo sed -i "s/Options Indexes FollowSymLinks/Options FollowSymLinks/" /etc/apache2/apache2.conf

Step 4: Start, enable and check the status of apache2

sudo systemctl stop apache2
sudo systemctl enable apache2
sudo systemctl start apache2

Install and Configure MariaDB

Step 1: Install Marais DB database server and client

sudo apt-get install mariadb-server mariadb-client -y

Step 2: Use the following commands to Start,  Stop, enable and check the status of MariaDB service.

sudo systemctl stop mysql
sudo systemctl start mysql
sudo systemctl enable mysql
sudo systemctl status mysql

Step 3: Setup the root admin password for the database using mysql_secure_installation command. It will prompt you for setting up the new password.

sudo mysql_secure_installation

Use the following options for the prompt.

Enter current password for root (enter for none): Just press the Enter
Set root password? [Y/n]: Y
New password: Enter password
Re-enter new password: Repeat password
Remove anonymous users? [Y/n]: Y
Disallow root login remotely? [Y/n]: Y
Remove test database and access to it? [Y/n]:  Y
Reload privilege tables now? [Y/n]:  Y

Step 4: Restart the database server for the changes to take effect.

sudo systemctl restart mysql

Setup and Configure PHP

Step 1: Install PHP 7 and its related modules.

sudo apt-get install php7.0 php7.0-mysql libapache2-mod-php7.0 php7.0-cli php7.0-cgi php7.0-gd

Step 2: Open  /etc/php/7.0/apache2/php.ini file

sudo vi /etc/php/7.0/apache2/php.ini

Add the following contents to the file.

file_uploads = On
allow_url_fopen = On
memory_limit = 256M
upload_max_filesize = 150M
max_execution_time = 350
date.timezone = America/Chicago

Configure Database For WordPress

Step 1: Login to the database using the password you set for root.

sudo mysql -u root -p

Step 2: Create a database named “wordpress-db”

CREATE DATABASE wordpress-db;

Step 3: Create a new databases user named “wordpress-admin” and set a custom password. This user will be used in the WordPress configuration.

CREATE USER 'wordpress-admin'@'localhost' IDENTIFIED BY 'your_password_here';

Step 4: Grant all privileged on wordpress-db for wordpress-admin user. Replace the password you set for wordpress-admin.

GRANT ALL ON wordpress-db.* TO 'wordpress-admin'@'localhost' IDENTIFIED BY 'your-password-here' WITH GRANT OPTION;

Step 5: Flush all privileges and exit the db shell.

FLUSH PRIVILEGES;
exit

Setup and Configure WordPress

Step 1: Download latest WordPress

wget https://wordpress.org/latest.tar.gz

Step 2: Untar the WordPress files

tar -xvf latest.tar.gz

Step 3: Move the WordPress folder to /var/www/html folder.

sudo mv wordpress /var/www/html/wordpress

Step 4: Change the ownership of the wordpress folder to www-data

sudo chown -R www-data:www-data /var/www/html/wordpress/

Step 5: Change the folder and file permissions using the following command. Folders should have 755 permission and files should have 644 permission.

find /var/www/html/ -type d -print0 | xargs -0 chmod 0755
find /var/www/html/ -type f -print0 | xargs -0 chmod 0644

You can check if the permissions have been applied using the following command.

stat -c "%a %n"  /var/www/html/wordpress/*

Configure Apache For WordPress

Step 1: Create a new Apache configuration named wp-site.conf for the WordPress site.

sudo vi /etc/apache2/sites-available/wp-site.conf

Add the following configuration to the file and save it. Replace yourdomain with you custom domain name.

<VirtualHost *:80>
     ServerAdmin [email protected]
     DocumentRoot /var/www/html/wordpress/
     ServerName yourdomain.com
     ServerAlias www.yourdomain.com

     <Directory /var/www/html/wordpress/>
        Options +FollowSymlinks
        AllowOverride All
        Require all granted
     </Directory>

     ErrorLog ${APACHE_LOG_DIR}/error.log
     CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

Step 2: Enable wp-site.conf

sudo a2ensite wp-site.conf

Step 3: Enable the apache rewrite module.

sudo a2enmod rewrite

Step 4: Restart the apache server.

sudo systemctl restart apache2

Configure WordPress

Step 1: Rename the default wp-config-sample.php to wp-config.php

sudo mv /var/www/html/wordpress/wp-config-sample.php /var/www/html/wordpress/wp-config.php

Step 2: Open the wp-config.php file

sudo vi /var/www/html/wordpress/wp-config.php

Replace the values highlighted in the image below with database, user, and password generated in previous steps.

wp-config configurations

Step 3: Now you can configure your site details by browsing the domain name you used in the wp-site.conf file. Make sure your server is mapped to the domain name you configures in the wp-site.conf file.

If you haven’t configured a domain name, you can access the installed WordPress with http://<your-server-ip>/wordpress path.

http://your-domain-name.com
[or]
http://<your-server-ip>/wordpress

 

Step 4: Select the preferred language and fill up the details in the configuration wizard.

wordpress configuration wizard

Step 5: Once WordPress is installed, you can access the dashboard using the username and password.

wordpress dashbaord

wordpress installation guide

How To Setup and Configure a Proxy Server – Squid Proxy

configure proxy server

A proxy server has many use cases. it could range from personal internet access to restrict organization systems/servers to access the external world or to limit external internet access for a set of servers on the cloud.

The best way to configure a proxy server is by using the Squid proxy. It is a widely used proxy server.

In this article, we have covered the following.

  1. Install proxy server
  2. Configure the proxy server
  3. Configure basic proxy authentication.

Note: This tutorial is tested on CentOS 7. For Ubuntu setup, check this tutorial – Squid Proxy Setup On Ubuntu

Install Proxy Server: Squid Proxy

Step1: Update the server

sudo yum update -y

Step 2: Configure EPEL repo.

sudo yum -y install epel-release
sudo yum -y update
sudo yum clean all

Step 3: Install squid

sudo yum -y install squid

Step 4: Start and enable squid server.

sudo systemctl start squid
sudo systemctl enable squid

Step 5: Check the status of squid server.

sudo systemctl status squid
squid server status

Configure Proxy Server: Squid Proxy

All the configurations for the squid server are present in /etc/squid/squid.conf file.

Configure proxy Sources To Access Internet

First, you need to configure the sources from which squid proxy should accept connections. For example, you might need to access this proxy server only from your home network or from specific CIDR ranges.

You can add a source IP range with an ACL using the following format.

acl localnet src 110.220.330.0/24

Open  /etc/squid/squid.conffile and add the source add as shown below. Change the IP to the desired network/IP source based on your needs. In the following example, we have added a single source IP.

squid server configuration

Restart the proxy server after making the ACL changes.

sudo systemctl restart squid

Test proxy Server Connectivity

Test if the proxy server is working using a simple curl request. Use the following curl format. By default squid proxy runs on 3128 port.

curl -x http://<squid-proxy-server-IP>:3128  -L http://google.com
/Users/bibin/Downloads/squid server connectivity test

Configure Proxy Authentication

Along with access ACL’s, you can add basic authentication to your proxy server for extra security. Follow the steps given below for setting up a basic auth for the squid proxy server.

Step 1: Install httpd-tools

 sudo yum -y install httpd-tools

Step 2: Create a passwd file and make squid as the file owner.

sudo touch /etc/squid/passwd && sudo chown squid /etc/squid/passwd

Step 3: Add pxuser  to the password file using htpasswd utility. It will prompt for a custom password. Enter a strong password you need. This username and password will be used for all connections through this proxy.

 sudo htpasswd /etc/squid/passwd pxuser

Step 4: Open squid config file.

sudo vi /etc/squid/squid.conf

Add the following to the config file and save it.

auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid/passwd
auth_param basic children 5
auth_param basic realm Squid Basic Authentication
auth_param basic credentialsttl 2 hours
acl auth_users proxy_auth REQUIRED
http_access allow auth_users

Step 5: Now, restart squid server for the configuration changes to take place.

sudo systemctl restart squid

Step 6: Now if you test the proxy connection using curl, you will get the “authentication required message” as shown below.

Now, test the connectivity with proxy user and password we configured in step 3. An example syntax is shown below.

curl -x http://35.196.101.43:3128  --proxy-user pxuser:12345  -I http://google.com

With username and password, your proxy request should go through.

Blocking Websites

Another great use of the proxy server is restricting the website access. Follow the steps below for creating a block list.

Step 1: Open a blocked list file.

sudo vi /etc/squid/blocked_sites

Add the websites to be blocked in the file. For example,

facebook.com
twitter.com
instagram.com

Step 2: Open the squid config file.

sudo vi /etc/squid/squid.conf

Add the following to the ACL list.

acl blocked_sites dstdomain "/etc/squid/blocked_sites"
http_access deny blocked_sites

Step 3: Restart the squid server.

sudo systemctl restart squid

Now if you try to access the blocked site through the proxy, you will get a forbidden message as shown below.

configure proxy server

How To Create Kubernetes Service Account For API Access

Setup Kubernetes API Access

The best way to have API access to the Kubernetes cluster is through service accounts. This tutorial will guide you through the process of creating the service account, role and role binding to have API access to the kubernetes cluster

Setup Kubernetes API Access Using Service Account

Follow the steps given below for setting up the API access using the service account.

Note: If you are using GKE on Google Cloud, you might need to run the following two commands to have access to create roles and role-bindings with your gcloud user.

ACCOUNT=$(gcloud info --format='value(config.account)')
kubectl create clusterrolebinding owner-cluster-admin-binding \
    --clusterrole cluster-admin \
    --user $ACCOUNT

Step 1: Create a service account named “api-service-account”

kubectl create serviceaccount api-service-account

Step 2: Create a “clusterRole.yaml” file and copy the following contents. You can also get this yaml file from here.

Note: This YAML declaration has a role with full access to all cluster resources and a role binding to “api-service-account”. It is not recommended to create a service account with all cluster component access. You can refer to the list of resources and verbs from this page

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: api-access
rules:
  -
    apiGroups:
      - ""
      - apps
      - autoscaling
      - batch
      - extensions
      - policy
      - rbac.authorization.k8s.io
    resources:
      - componentstatuses
      - configmaps
      - daemonsets
      - deployments
      - events
      - endpoints
      - horizontalpodautoscalers
      - ingress
      - jobs
      - limitranges
      - namespaces
      - nodes
      - pods
      - persistentvolumes
      - persistentvolumeclaims
      - resourcequotas
      - replicasets
      - replicationcontrollers
      - serviceaccounts
      - services
    verbs: ["*"]
  - nonResourceURLs: ["*"]
    verbs: ["*"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: api-access
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: api-access
subjects:
- kind: ServiceAccount
  name: api-service-account
  namespace: default

Step 3: Get the secret name associated with the api-service-account

kubectl get serviceaccount api-service-account  -o json | jq -Mr '.secrets[].name'

Step 4: Now, use the secret name you got in step 4 to get the base64 decoded token.

kubectl get secrets <name-of-the-secret> -o json | jq -Mr '.data.token' | base64 -D

For example,

kubectl get secrets api-service-account-token-cpf5f  -o json | jq -Mr '.data.token' | base64 -D

Step 5: Get the cluster endpoint to check the API access. The following command will display the cluster endpoint (IP, DNS).

kubectl get endpoints | grep kubernetes

Step 6: Now that you have the cluster endpoint and token for the service account, you can test the API connectivity using CURL or postman app.

For example,

curl -k  https://35.226.193.217/api/v1/namespaces -H "Authorization: Bearer eyJhbGcisdfsdfsdfiJ9.eyJpc3MiOisdfsdfVhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3sdf3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFwaS1zZXJ2aWNlsdfglkjoer876Y3BmNWYiLsdfsdfRlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0OmFwaS1zZXJ2aWNlLWFjY291bnQifQ.u5jgk2px_lEs3f5e5lh_UfS40fndtDKMTY5UvsdfrtsuhtgjrUj-ezrRXeLS8SLOae4DuOGGGbInSg_gIo6oO7bLHhCixWOBJNOA5gzrLVioof_kHDR8gH5crrsWoR-GSSsdfgsdfg6fA_LDOqdxzqMC0WlXt6tgHfrwIHerPPvkI6NWLyCqX9tn_akpcihd-bL6GwOKlph17l_ND710FnTkE7kBfdXtQWWxaPPe06UEmoKK9t-0gsOCBxJxViwhHkvwqetr987q9enkadfgd_2cY_CA"

ONLINE COURSE: The Complete Kubernetes Course

Learn how you can run, deploy, manage and maintain containerized Docker applications on Kubernetes

  • Learn to launch kubernetes cluster
  • Get started with Containerization of apps
  • Deploy applications on kubernetes cluster
  • Run stateful and stateless applications on containers
Setup Kubernetes API Access