How To Install Latest Sonatype Nexus 3 on Linux

Install Latest Sonatype Nexus 3 on Linux

Sonatype Nexus is one of the best repository managers out there. It is some tool that you cannot avoid in your CI/CD pipeline. It effectively manages deployable artifacts.

Sonarqube requirements

  1. Minimum 1 VCPU & 2 GB Memory
  2. Server firewall opened for port 22 & 8081
  3. OpenJDK 8
  4. All Nexus process should run as a non-root nexus user.

Note: For production setup, please consider minimum production hardware requirements based on the nexus usage and data storage. Checkout the official system requirements document for detailed information

Sonatype Nexus 3 on Linux ec2

This article guides you to install and configure Sonatype Nexus 3 in a secure way on an ec2 Linux System.

Note: This was tested on a Redhat machine and it will work on Centos or related Linux flavours as well.

Step 1: Login to your Linux server and update it. Also install required utilities.

sudo yum update -y
sudo yum install wget -y

Step 2: Install OpenJDK 1.8

sudo yum install java-1.8.0-openjdk.x86_64 -y

Step 3: Create a directory named app and cd into the directory.

sudo mkdir /app && cd /app

Step 4: Download the latest nexus. You can get the latest download links fo for nexus from here.

sudo wget -O nexus.tar.gz https://download.sonatype.com/nexus/3/latest-unix.tar.gz

Untar the downloaded file.

sudo tar -xvf nexus.tar.gz

Rename the untared file to nexus.

sudo mv nexus-3* nexus

Step 5: As a good security practice, it is not advised to run nexus service with any sudo user. So create a new user named nexus.

sudo adduser nexus

Change the ownership of nexus files and nexus data directory to nexus user.

sudo chown -R nexus:nexus /app/nexus
sudo chown -R nexus:nexus /app/sonatype-work

Step 6: Open /app/nexus/bin/nexus.rc file

sudo vi  /app/nexus/bin/nexus.rc

Uncomment run_as_user parameter and set it as following.

run_as_user="nexus"

Step 7: If you want to change the default nexus data directory, open nexus properties file and change the data directory “-Dkaraf.data” parameter to a preferred location as shown below. If you dont specify anything, by default nexus data directory will be set to /app/sonatype-work/nexus3

Tip: For production setup, it is is always better to mount the nexus data directory to a separate data disk attached to the server. So that backup and restore can be done easily.

sudo vi /app/nexus/bin/nexus.vmoptions

An example configuration is shown below.

-Xms2703m
-Xmx2703m
-XX:MaxDirectMemorySize=2703m
-XX:+UnlockDiagnosticVMOptions
-XX:+UnsyncloadClass
-XX:+LogVMOutput
-XX:LogFile=../sonatype-work/nexus3/log/jvm.log
-XX:-OmitStackTraceInFastThrow
-Djava.net.preferIPv4Stack=true
-Dkaraf.home=.
-Dkaraf.base=.
-Dkaraf.etc=etc/karaf
-Djava.util.logging.config.file=etc/karaf/java.util.logging.properties
-Dkaraf.data=/nexus/nexus-data
-Djava.io.tmpdir=../sonatype-work/nexus3/tmp
-Dkaraf.startLocalConsole=false

Running Nexus as a System Service

It is better to have systemd entry to manage nexus using systemctl. Follow the steps given below for the setup.

Create a nexus systemd unit file.

sudo vi /etc/systemd/system/nexus.service

Add the following contents to the unit file.

[Unit]
Description=nexus service
After=network.target

[Service]
Type=forking
LimitNOFILE=65536
User=nexus
Group=nexus
ExecStart=/app/nexus/bin/nexus start
ExecStop=/app/nexus/bin/nexus stop
User=nexus
Restart=on-abort

[Install]
WantedBy=multi-user.target

Manage Nexus Service

Now we have all the configurations in place to run nexus.

Execute the following command to add nexus service to boot.

sudo chkconfig nexus on

To start the Nexus service, use the following command.

sudo systemctl start nexus

The above command will start the nexus service on port 8081. To access the nexus dashboard, visit http://:8081. You will be able to see the nexus homepage as shown below.

nexus 3 homepage

To log in, use the default username and password.

Default username is admin

You can find the default admin password in /app/sonatype-work/nexus3/admin.password file.

cat /app/sonatype-work/nexus3/admin.password

For stopping,

sudo systemctl stop nexus

For restarting,

sudo systemctl restart nexus

You Might Like: How To Setup Latest Nexus OSS On Kubernetes

Install Latest Sonatype Nexus 3 on Linux

How To Setup Consul Cluster (Multi-Node) on Linux – Beginners Guide

Setup Consul Cluster (Multi-Node) on Linux

Consul is an open source key-value store. It is used for use cases such as service discovery, config management, etc. This guide has detailed instructions to set up a consul cluster with multiple nodes.

Prerequisites

  1. Three Linux servers
  2. Following ports opened between all three servers. If you on AWS, Azure or GCP make sure you have the security groups and firewall tags added properly to allow communications of the below-mentioned ports.
    • 8300  – TCP
    • 8301  – TCP & UDP
    • 8302  – TCP & UDP
    • 8400  – TCP
    • 8500  – TCP
    • 8600  – TCP & UDP

Setup Consul Cluster

This tutorial is based on a three-node consul cluster. The nodes are named as follows.

  1. consul-1
  2. consul-2
  3. consul-3

Follow the steps given below for a fully functional consul cluster.

Install and Configure Consul on All the Three Nodes

The following steps have to be performed on all the three nodes except step 4.

Step 1: CD into bin directory and download Linux consul binary from here

cd /usr/local/bin
sudo curl -o consul.zip https://releases.hashicorp.com/consul/1.6.0/consul_1.6.0_linux_amd64.zip

Step 2: Unzip the downloaded file and remove the zip file.

unzip consul.zip
sudo rm -f consul.zip

Step 3: Create the following two directories.

sudo mkdir -p /etc/consul.d/scripts
sudo mkdir /var/consul

Step 4: Create a consul secret using the following command from one of the three servers. Copy the secret to a text file.

consul keygen

Step 5: Create a config file on all three servers.

sudo vi /etc/consul.d/config.json

Copy the following config to the file. Replace encrypt value with the secret created in step 4 and start_join IP’s with your server IP’s.

{
    "bootstrap_expect": 3,
    "client_addr": "0.0.0.0",
    "datacenter": "Us-Central",
    "data_dir": "/var/consul",
    "domain": "consul",
    "enable_script_checks": true,
    "dns_config": {
        "enable_truncate": true,
        "only_passing": true
    },
    "enable_syslog": true,
    "encrypt": "goplCZgdmOFMZ2Q43To0jw==",
    "leave_on_terminate": true,
    "log_level": "INFO",
    "rejoin_after_leave": true,
    "server": true,
    "start_join": [
        "10.128.0.2",
        "10.128.0.3",
        "10.128.0.4"
    ],
    "ui": true
}

Create a Consul Service

Execute the following steps on all the three nodes.

Step 1: Create a systemd file.

sudo vi /etc/systemd/system/consul.service

Copy the following contents to the file.

[Unit]
Description=Consul Startup process
After=network.target

[Service]
Type=simple
ExecStart=/bin/bash -c '/usr/local/bin/consul agent -config-dir /etc/consul.d/'
TimeoutStartSec=0

[Install]
WantedBy=default.target

Step 2: Reload the system daemons

sudo systemctl daemon-reload

Bootstrap and Start the Cluster

Step 1: On consul-1 server, start the consul service

sudo systemctl start consul

Step 2: Start consul on other two servers (Consul-2 and consul-3) using the following command.

sudo systemctl start consul

Step 3: Check the cluster status by executing the following command.

/usr/local/bin/consul members

You should get an output like the following. It means your consul cluster is up and running.

[[email protected] ~]$ /usr/local/bin/consul members
Node      Address          Status  Type    Build  Protocol  DC          Segment
consul-1  10.128.0.2:8301  alive   server  1.2.0  2         us-central  <all>
consul-2  10.128.0.3:8301  alive   server  1.2.0  2         us-central  <all>
consul-3  10.128.0.4:8301  alive   server  1.2.0  2         us-central  <all>

Access Consul UI

From consul version 1.20, UI is an inbuilt consul component.

You can access the consul web UI using the following URL syntax.

http://<consul-IP>:8500/ui

For example,

http://35.238.163.87:8500/ui

You can view the UI as shown below.

consul web ui setup

Also, you can view a complete UI demo from here

Other Consul Blog Series,

  1. Setup And Configure Consul Agent On Client Mode
Setup Consul Cluster (Multi-Node) on Linux

Free Kubernetes Ebook: Kubernetes up and running

Free Kubernetes Ebook

If you are looking for a Kubernetes book to get started or to learn all the Kubernetes concepts, Kubernetes up and running is for you. Even if you have an understanding of Kubernetes, we can guarantee that you will learn new things.

About the Authors:

This book is authored by the following three authors.

  1. Brendan Burns
  2. Joe Beda
  3. Kelsey Hightower

Authors of this book are involved in Kubernetes development since its inception and well-known names in the tech industry.

What is in the Book?

This book is close to 235 pages. It covers all the Kubernetes concepts in detail. Authors have done a great job in explaining all the concepts in an easy way.

Here are the key things covered in the book

  1. Introduction to Kubernetes & Why Kubernetes [Velocity, Scaling, Infrastrucutre]
  2. Key container concepts [Images, Docker, Multistage Build, Container Security]
  3. Kubernetes Cluster Setup [GKE, AKS, EKS, Minikube]
  4. Kubernetes CLI commands [kubectl]
  5. Kubernetes Objects Deep Dive [ Pods, Deployments, Replicasets, Deamonsets, jobs, Labels & annotations etc]
  6. Kubernetes Service Discovery
  7. Load balancing with Ingress
  8. Kubernetes RBAC
  9. Kubernetes Storage
  10. Running stateful sets on Kubernetes
  11. Kubernetes Custom Resources and Operator patterns.
  12. Realworld application deployments [Jupyter, Ghost, Redis]

Kubernetes up and running Free Ebook

Microsoft has done a great job by making the second edition ebook completely free.

You Might Like: Kubernetes CKA/CKAD Certification Coupon

Other Kubernetes Books

The following are the best Kubernetes books you can read.

  1. Kubernetes Up and running [2nd Edition Paperback]
  2. Kubernetes in Action
  3. Kubernetes cookbook

Other Kubernetes Resources

Following are the best resources you can use to get your journey started with Kubernetes

  1. Kubernetes for the Absolute Beginners – Hands-on Course
  2. Kubernetes Certification Tips from a Kubernetes Certified Administrator
  3. Learn Kubernetes using Interactive Browser
  4. Kubernetes The Hard Way

If you want to get started with the Kubernetes clusters, start with GKE or any other managed Kubernetes services.

Free Kubernetes Ebook

List of Best Open Source Service Discovery Tools

List of Best Open Source Service Discovery Tools

With evolving microservice architecture, service discovery is becoming a must-have for all modern applications. New components that get deployed should be able to find other service endpoints very efficiently without much latency. Service discovery paves a way for this. You can read about service discovery from here (What does service discovery really mean)

Microservices and service discovery go hand in hand and the following open source tools that provide service discovery functionality.

1. Consul

Consul is a tool for service discovery, monitoring, and configuration. It uses Serf to form dynamic clusters and a peer-to-peer data store, based on the Serf library. Consul is a highly distributed service discovery tool.

You Might Like: How To Setup Consul Cluster

It can act as a key-value store for configuration management. Serf (gossip protocol) is used to manage everything in the cluster like failure detection. Another consensus protocol called Raft manages consistency within the system.

Consul Features:

  1. Consul makes it easy for services to register themselves and to discover other services by MySQL, DNS or HTTP interface.
  2. The DNS support is extensive and they can be configured to make the process more seamless.
  3. Consul offers excellent health check features that can alert the operators of any issue in the cluster.
  4. Consul also offers a key/value storage that allows dynamic configurations, feature flagging and a lot more.
  5. It has HTTP APIs to store and retrieve key/value data in a distributed key/value store.

Consul can, however, be a bit daunting as the distributed systems including Consul have an inherent complexity. So, it’s not really a problem of Consul but more of a general problem. Users don’t need to implement their own third party library as Consul comes with its own library. Consul has a similar concept like the Netflix OSS Sidecar concept that allows non-Zookeeper clients to register and remain discoverable.

Companies using Consul include DigitalOcean, EverythingMe, Percolate, Outbrain, SendGrid and more!

2. Etcd

Etcd, a core component of CoreOS is another tool offering key/value store functionality. It is similar to both Zookeeper and Consul. Developed in the Go language, it uses Raft just like Consul for consensus. It provides a fast and reliable HTTP and JSON based API with a query and pushes notification.

Typically three, five or seven nodes are present in the cluster. Etcd can be used in microservices architectures where the containers need both service registration and service discovery, i.e., to write the key-value pairs for registration and reading the key-value pairs for service discovery.

You Might Like: How to Setup an etcd cluster

Other applications can also utilize the etcd by using a project called confd to convert the information stored in etcd into static configuration files. The clients need to manage any connection failure and re-connect with another service instance.

Etcd is being used by companies like Google, Kubernetes, Cloud Foundry, Red Hat, Zenreach, Headspace, Apptus, CloudGear and more. Over the years, technology and community support for etcd has evolved and offers a good experience to developers.

3. Apache Zookeeper

Apache ZooKeeper is a distributed, centralized and consistent service. Written in Java language, it uses the Zab protocol to manage changes in a cluster. It emerged out of the Hadoop world where it had the role to maintain components in a cluster. The data is stored in a hierarchical namespace, in a file system or a tree.

The nodes exist as long as the client is connected to the network and if the network disconnects, the node will also disappear. The clients have to handle load balancing or any failure. They also receive registered services, notification whenever new services register.

You Might Like: How To Setup a Zookeeper Cluster

Although Zookeeper is a consistent system, there can be failures in the system – some systems won’t be able to register or the read and write function might return an error. It is a robust, old and established application with a huge and vibrant community, clients and extensive library support.

ZooKeeper is used by companies including Rackspace, Yahoo!, Reddit, eBay, Solr, Cloudera, Luxoft, F5 Networks, Apache Software Foundation, Spero Solutions to name a few. It has a market share of nearly 0.4%.

Conclusion

Service discovery had become an integral part of all infrastructure implementation to achieve high availability, failure detection and much more.

What type of service discovery mechanism are you using pr planning to use in your infrastructure?

List of Best Open Source Service Discovery Tools

How to Setup Custom UI Theme For Jenkins

Setup Custom UI Theme For Jenkins

If you are bored with the old Jenkins UI, its font, and icons, you can give your Jenkins a makeover using custom CSS styling with a custom logo.

Custom CSS Main Features:

  1. Flat UI Fonts
  2. Better Syntax highlighting for Shell blocks
  3. Better highlighted console output

Modifying Jenkins UI

Follow this tutorial for changing the look and feel of default Jenkins UI.

Step 1: Go to Manage Jenkins –> Manage Plugins. Click available tab and search for simple theme plugin.

Step 2: Install the theme and restart Jenkins.

Step 3: Go to Manage Jenkins –> Configure and search for Theme configuration and in the CSS field enter the following URL and save it.

https://cdn.rawgit.com/afonsof/jenkins-material-theme/gh-pages/dist/material-cyan.css
custom jenkins css

Once it is saved, the UI will be changed to a materialized flat UI based on cyan color.

materialized jenkins ui

Uploading Custom CSS TO Jenkins Server

In the above example, we are referring to a CSS file hosted in third party website. You can also host this CSS on your Jenkins server. Follow the steps given below.

You Might Like: Jenkins 2 Tutorial For Beginners – Getting Started Guide

Step 1: Login to your jenkins server, and cd into your Jenkins home directory.

Step 2: Create a folder named layout inside the userContent directory.

cd userContent
mkdir layout

Step 3: cd into the layout directory and create a style.css file.

cd layout
vi style.css

Step 4: Now, visit the following URL in the browser and copy the whole CSS content and paste it in the style.css file and save it.

https://cdn.rawgit.com/afonsof/jenkins-material-theme/gh-pages/dist/material-cyan.css

The following are the colors supported. You can replace the color name at the end in the above URL to have the desired color

jenkins custom UI Colors

Step 4: Now, under Manage Jenkins –> Configure, under the theme section, replace the full URL with the following.

/userContent/layout/style.css

More Customization

You can customize the looks more by changing the logo. If you want more customizations and colors, you can follow the office Jenkins materialize CSS site from here.

You can check out the video tutorial for this article.

Setup Custom UI Theme For Jenkins

How to Install and configure Sonarqube on Linux (RHEL/Centos/ec2)

Install and Configure Sonarqube on Linux

Sonarqube is a great tool for source code quality management, code analysis etc. This is the most widely used tool for code coverage and analysis.

Install and  Configure Sonarqube on Linux

This guide will help you to set up and configure sonarqube on Linux servers (Redhat/Centos 7 versions) on any cloud platforms like ec2, azure, compute engine or on-premise data centers. Follow the steps given below for the complete sonarqube configuration.

Sonarqube requirements

  1. Server with minimum 2GB/1 vcpu capacity
  2. PostgreSQL version 9.3 or greater.
  3. OpenJDK 11 or JRE 11
  4. All sonarquber process should run as a non-root sonar user.

You can find the official requirement doc here.

Update: MySQL for Sonarqube is depricated

Prep the Server With Required Softwares

Step 1: Update the server.

sudo yum update -y

Step 2: Install wget & unzip

sudo yum install wget unzip -y

Step 3: Install java 11

sudo yum install java-11-openjdk-devel -y

Step 4: Login as root and execute the following commands.

sysctl vm.max_map_count
sysctl fs.file-max
ulimit -n
ulimit -u

Setup PostgreSQL 10 Database For SonarQube

Step 1: Install PostgreSQL 10 repo.

sudo yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-centos10-10-2.noarch.rpm -y

Step 2: Install PostgreSQL 10

sudo yum install postgresql10-server postgresql10-contrib -y

Step 3: Initialize the database.

sudo /usr/pgsql-10/bin/postgresql-10-setup initdb

Step 4: Open /var/lib/pgsql/data/pg_hba.conf file to change the authentication to md5.

sudo vi /var/lib/pgsql/10/data/pg_hba.conf

Find the following lines at the bottom of the file and change peer to trust and idnet to md5

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     peer
# IPv4 local connections:
host    all             all             127.0.0.1/32            ident
# IPv6 local connections:
host    all             all             ::1/128                 ident

Once changed, it should look like the following.

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            md5
# IPv6 local connections:
host    all             all             ::1/128                 md5

Step 5: Start and enable PostgreSQL.

sudo systemctl start postgresql-10
sudo systemctl enable postgresql-10

Step 6: You can verify the installation using the following version select query.

sudo -u postgres /usr/pgsql-10/bin/psql -c "SELECT version();"

Setup Sonar User and Database

We need to have a sonar user and database for the sonar application.

Step 1: Change the default password of the Postgres user. All Postgres commands have to be executed from this user.

sudo passwd postgres

Step 2: Login as postgres user with the new password.

su - postgres

Step 3: Login to the PostgreSQL CLI.

psql

Step 4: Create a sonarqubedb database.

create database sonarqubedb;

Step 5: Create the sonarqube DB user with a strongly encrypted password. Replace your-strong-password with a strong password.

create user sonarqube with encrypted password 'your-strong-password';

Step 6: Next, grant all privileges to sonrqube user on sonarqubedb.

grant all privileges on database sonarqubedb to sonarqube

Step 7: Exit the psql prompt using the following command.

\q

Step 6: Switch to your sudo user using the exit command.

exit

Setup Sonarqube Web Server

Step 1: Download the latest sonarqube installation file to /opt folder. You can get the latest download link from here. http://www.sonarqube.org/downloads/

cd /opt 
sudo wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-7.6.zip

2. Unzip sonarqube source files and rename the folder.

sudo unzip sonarqube-7.6.zip
sudo mv sonarqube-7.6 sonarqube

4. Open /opt/sonarqube/conf/sonar.properties file.

sudo vi /opt/sonarqube/conf/sonar.properties

Uncomment and edit the parameters as shown below. Change the password accordingly. You will find jdbc parameter under PostgreSQL section.

sonar.jdbc.username=sonar                                                                                                                     
sonar.jdbc.password=sonar-db-password
sonar.jdbc.url=jdbc:postgresql://localhost/sonar

By default, sonar will run on 9000. If you want on port 80 or any other port, change the following parameters for accessing the web console on that specific port.

sonar.web.host=0.0.0.0
sonar.web.port=80

If you want to access sonarqube some path like http://url:/sonar, change the following parameter.

sonar.web.context=/sonar

Add Sonar User and Privileges

Create a user named sonar and make it the owner of the /opt/sonarqube directory.

sudo useradd sonar
sudo chown -R sonar:sonar /opt/sonarqube

Start Sonarqube Service

To start sonar service, you need to use the script in sonarqube bin directory.

Step 1: Login as sonar user

sudo su - sonar

Step 2: Navigate to the start script directory.

cd /opt/sonarqube/bin/linux-x86-64 

Step 3: Start the sonarqube service.

./sonar.sh start

Now, you should be able to access sonarqube on the browser on port 9000

Step 4: Check the application status. If it is in running state, you can access the sonarqube dashboard using the DNS name or Ip address of your server.

sudo ./sonar.sh status

Setting up Sonarqube as a service

Step 1: Create a file /etc/systemd/system/sonarqube.service

sudo vi /etc/systemd/system/sonarqube.service

Step 2: Copy the following content on to the file.

[Unit]
Description=SonarQube service
After=syslog.target network.target

[Service]
Type=simple
User=sonarqube
Group=sonarqube
PermissionsStartOnly=true
ExecStart=/bin/nohup java -Xms32m -Xmx32m -Djava.net.preferIPv4Stack=true -jar /opt/sonarqube/lib/sonar-application-7.6.jar
StandardOutput=syslog
LimitNOFILE=65536
LimitNPROC=8192
TimeoutStartSec=5
Restart=always

[Install]
WantedBy=multi-user.target

Step 3: Start and enable sonarqube

sudo systemctl start sonarqube
sudo systemctl enable sonarqube

Step 4: Check the sonarqube status to ensure it is running as expected.

sudo systemctl status  sonarqube

Troubleshooting Sonarqube

All the logs of sonarqube are present in the /opt/sonarqube/logs directory.

cd /opt/sonarqube/logs

You can find the following log files.

es.log
sonar.log
web.log
access.log

Using tail command you can check the latest logs. For example,

tail -f access.log

For sonarqube support, visit this link

In this tutorial, we covered how to install and configure sonarqube on a Linux server. If you face any issues during the setup, please feel free to leave a comment below.

Install and Configure Sonarqube on Linux