This blog will discuss setting up a Highly available Jenkins using the open-source version.
Before getting into the high availability discussion, you first need to understand how Jenkins manages its data and the high-level architecture.
If you are not aware of the architecture, I highly suggest reading Jenkins architecture post.
Types of Jenkins HA Setup
There is no direct way of setting up Jenkins in High availability mode due to how Jenkins manages its data.
Jenkins manages its data in a flat-file (xml), and you can have only one instance of the master node that can read the Jenkins data.
Let’s say, if you try to configure multiple master nodes with a shared volume, you will have inconsistent read and writes that lead to unstable Jenkins.
There are two ways you can have Jenkins in HA mode.
- Jenkins active/passive setup: Only enterprise Jenkins comes with a supported plugin to have this setup. But most organizations use open-source versions, and his option is out of scope. Also there is an option with HAproxy, however I think its bit of an administrative overhead.
- Jenkins HA setup in autoscaling Group: Having Jenkins in autoscaling group is workaround to have a higly available Jenkins. It is not a 100% HA solution; however, for any reason if your Jenkins server crasher, another instance will come up within a short period.
In this blog, we will discuss the HA setup using autoscaling.
Jenkins HA Setup With Autoscaling Group
You can call this method a poor man’s solution for Jenkins HA. Becuase, when a instance goes down, there is few minutes of downtime for other instance to come up (vm + java service startup time). But it works 🙂
This method will work on any private/public cloud platform with the option of autoscaling.
This method is similar to deploying Jenkins on Kubernetes, where if the Jenkins pod goes down, another pod will come up with the same data.
Instead of a pod, it’s a VM., so the startup time takes more time than spinning up a new pod.
Let’s look at the high-level architecture of the Jenkins HA setup using an autoscaling group.
Here the concept is pretty simple.
- Jenkins is deployed in an autoscaling group with min and max one count. It ensures we have only one instance of Jenkins running all the time.
- Then, we have a dedicated external disk to hold the Jenkins data. I could be a separate volume or an NFS share. The autoscaling image startup script (AMI user data) gracefully attaches this disk when it comes up.
- If the active Jenkins instance goes down, the autoscaling policies will bring up another instance while terminative the inactive one. In this process, the data disk gets detached from the terminating instance gracefully (in AWS using lifecycle hooks) and gets attached to the new instance, preserving the previous Jenkins state and data.
- All the existing jobs will fail during the downtime or continue when the new instance comes up.
This setup also makes the upgrade and pathcing process so easy.
Do try out this setup and see if it works for you.
Also, you should backup jenkins data periodically using plugins and disk snapshots. You can easily restore a working version if you have all the data backup.
If you are new to Jenkins, checkout our Jenkins tutorial for beginners.