Jenkins Architecture Explained – Beginners Guide

Jenkins architecture and core concepts

Jenkins is an easy to use opensource CI/CD tool. This blog covers the fundamental Jenkins components architecture. If you are a beginner to Jenkins, it will help you gain some idea on how Jenkins components work together and the key configurations involved.

Jenkins Architecture

The following diagram shows the overall architecture of Jenkins.

Jnekins architecture explained

Following are the key components in Jenkins

  1. Jenkins Master Node
  2. Jenkins Slave Node
  3. Jenkins Web Interface

Let look at each component in detail.

Jenkins Master (Server)

Jenkins’s server or master node holds all key configurations.

The following are the key Jenkins master components.

  1. Jenkins Jobs: A job is a collection of steps that you can use to build your source code, test your code, run a shell script, or to run an Ansible role in a remote host. Ther are multiple job types available to support your workflow for continuous integration & continuous delivery.
  2. Jenkins Plugins: Plugins are community-developed modules that you can install in your Jenkins server. It lets you add more functionalities that are not natively available in Jenkins. You can also develop your custom plugins. Check out all plugins from the Jenkins Plugin Index
  3. Jenkins User: Jenkins has its own user database. It can be used for Jenkins’s authentication.
  4. Jenkins Global Security: Jenkins has the following two primary authentication methods.
    1. Jenkins’s own user database:- Set of users maintained by Jenkins’s own database.
    2. LDAP Integration:- Jenkins authentication using corporate LDAP configuration.
  5. Jenkins Credentials: If you want to save any secret information that has to be used in the jobs, you can store it as a credential. All credentials are encrypted by Jenkins.
  6. Jenkins Nodes/Clouds: You can configure multiple slave nodes (Linux/Windows) or clouds (docker, kubernetes) for executing Jenkins jobs.
  7. Jenkins Global Settings (Configure System): Under Jenkins global configuration, you have all the configurations of installed plugins and native Jenkins global configurations. Also, you can configure global environment variables under this section.
  8. Jenkins Logs: Provides logging information on all Jenkins server actions including job logs, plugin logs, webhook logs, etc.

All the configurations for the above-mentioned components will be present as a config file in the Jenkins master node.

Note: Jenkins doesn’t have a database. All Jenkins configurations are stored as flat config files. Mostly xml files.

Jenkins Slave

Jenkins slaves are the worker nodes for the jobs configured in Jenkins server.

Note: You can run jobs in Jenkins server without a Jenkins slave. However, the recommended approach is to have segregated Jenkins Slaves for different job requirements so that you don’t end up messing up the Jenkins server for any system wide configuration changes required for a job.

You can have any number of Jenkins slaves attached to a master with a combination of Windows & Linux servers.

Also, you can restrict jobs to run on specific slaves, depending on the use case. For example, if you have a slave with java 8 configurations, you can assign this slave for jobs that require Java 8 environment.

There is no single standard for using the slaves. You can set up a workflow and strategy based on your project needs.

Jenkins Web Interface

Jenkins 2.0 introduced a very intuitive web interface called “Jenkins Blue Ocean”. It has a good visual representation of all the pipelines.

Jenkins Master-Slave Connectivity

You can connect a Jenkins master and slave in two ways

  1. Using the SSH method: Uses the ssh protocol to connect to the slave. The connection gets initiated from the Jenkins master. Ther should be connectivity over port 22 between master and slave.
  2. Using the JNLP method: Uses java JNLP protocol. In this method, a java agent gets initiated from the slave with Jenkins master details. For this, the master nodes firewall should allow connectivity on specified JNLP port. Typically the port assigned will be 50000. This value is configurable.

There are two types of Jenkins slaves

  1. Slave Nodes: These are servers (Windows/Linux) that will be configured as static slaves. These slaves will be up and running all the time and stay connected to the Jenkins server. Organizations use custom scripts to shut down and restart the slaves when is not used. Typically during nights & weekends.
  2. Slave Clouds: Jenkins Cloud slave is a concept of having dynamic slaves. Means, whenever you trigger a job, a slave will be deployed as a VM/container on demand and gets deleted once the job is completed. This method saves money in terms of infra cost when you have a huge Jenkins ecosystem and continuous builds.

Jenkins Data

All the Jenkins data will be store in the following folder location. Data includes all jobs config files, plugins configs, secrets, node information, etc.

/var/lib/jenkins/

It is very important to back up the Jenkins data folder every day. For some reason, if your Jenkins server data gets corrupt, you can restore whole Jenkins with the data backup.

Jenkins architecture and core concepts

How to Attach and Mount an EBS volume to EC2 Linux Instance

Attach and Mount an EBS volume to EC2 Instance

AWS allows you to create new EBS volumes and you can attach it to instances for extra storage. However, to make it usable as storage inside the instance, you need to mount it to a specific folder.

Mount an EBS volume to EC2 Linux

In this tutorial, we will teach you how to attach and mount an EBS volume to ec2 Linux instances.

Follow the steps given below carefully for the setup.

Step 1: Head over to EC2 –> Volumes and create a new volume of your preferred size and type.

Note: Make sure the EBS volume and the instance are in the same zone.

Step 2: Select the created volume, right-click and select the “attach volume” option.

Step 3: Select the instance from the instance text box as shown below.

attach ebs volume

Step 4: Now, login to your ec2 instance and list the available disks using the following command.

 lsblk

The above command will list the disk you attached to your instance.

Step 5: Check if the volume has any data using the following command.

sudo file -s /dev/xvdf

If the above command output shows “/dev/xvdf: data“, it means your volume is empty.

Step 6: Format the volume to the ext4 filesystem using the following command.

sudo mkfs -t ext4 /dev/xvdf

Step 7: Create a directory of your choice to mount our new ext4 volume. I am using the name “newvolume”

sudo mkdir /newvolume

Step 8: Mount the volume to “newvolume” directory using the following command.

sudo mount /dev/xvdf /newvolume/

Step 9: cd into newvolume directory and check the disk space for confirming the volume mount.

cd /newvolume
df -h .

The above command would show the free space in the newvolume directory.

To unmount the volume, you have to use the following command.

umount /dev/xvdf

Get 95% Off Today: Best AWS Certified Solutions Architect – Associate Course 

Automount EBS Volume on Reboot

By default on every reboot, the  EBS volumes other than root volume will get unmounted. To enable automount, you need to make an entry in the /etc/fstab file.

Step 1: Back up the /etc/fstab file.

sudo cp /etc/fstab /etc/fstab.bak

Step 2: Open /etc/fstab file and make an entry in the following format.

device_name mount_point file_system_type fs_mntops fs_freq fs_passno

For example,

/dev/xvdf       /newvolume   ext4    defaults,nofail        0       0

Step 3: Execute the following command to check id the fstab file has any error.

sudo mount -a

If the above command shows no error, it means your fstab entry is good.

Now, on every reboot, the extra EBS volumes will get mounted automatically.

Add Multiple EBS Volumes To EC2 Instance

You can add more than one EBS volumes to an ec2 instance as an additional disk. You can follow the same steps explained above and the only difference will be the disk name that you get from the lsblk command.

Here is an example output having two EBS volumes attached to an instance.

Multiple EBS volumes on ec2 instance

How Many EBS volumes can I attach to an Instance?

You can attach a maximum of 20 EBS volumes to the instance. However, limit the maximum volumes to have optimal performance. Also, plan your instance capacity based on the workload you are trying to run.

For example, databases require high IOPS for high read-write rates. IOPS depends on the disk size. Higher the size, the higher the IOPS.

Also, it is recommended to take snapshots of EBS volumes for high data high availability and restore option. Check out EBS snapshot automation article for strategies and restore options.

That’s how you mount and unmount EBS volumes in your ec2 instances. If you get any error during the setup, please feel free to contact us in the comment section.

Attach and Mount an EBS volume to EC2 Instance