The resource utilization of the Jenkins slaves will be very less if you do not have builds happening continuously. In this scenario, it is better to use ephemeral Docker containers as Jenkins build slaves for better resource utilization. As you know, spinning up a new container takes less than a minute, every build spins up a new container, builds the project and will get destroyed. This way you can reduce the number of static Jenkins build VMs.
Docker containers as Build Slaves
In this guide, I will walk you through the steps for configuring docker container as build slaves.
I assume that you have a Jenkins server up and running. If you do not have one, follow this tutorial. How to setup Jenkins 2
If you want docker based Jenkins setup, you can follow this tutorial -> Setup Jenkins On a Docker container
Let’s Implement It
The first thing we should do is set up a docker host. Jenkins server will connect to this host for spinning up the slave containers. I am going to use the Centos server as my docker host. You can use any OS which supports Docker.
[Important] Configure a Docker Host With Remote API
- Spin up a VM, and install docker on it. You can follow the official documentation for installing docker. based on the Linux distribution you use.
- Jenkins master connects to the docker host using REST API’s. So we need to enable the remote API for our docker host. Follow enabling docker remote API tutorial to enable Docker remote API.
Once you enabled and tested the API, you can now start configuring the Jenkins server.
Create a Jenkins Slave Docker Image
Next step is to create a slave image.
If you are creating the image on your own, it image should contain the following minimum configurations to act as a slave.
- sshd service running on port 22.
- Jenkins user with password.
- All the required application dependencies for the build. For example, for a java maven project, you need to have git, java, and maven installed on the image.
Make sure sshd service is running and can be logged into the containers using a username and password. Otherwise, Jenkins will not be able to start the build process.
Note: The default ssh username is jenkins and password is also jenkins as per the given Dockerfile. You will have to use these credentials in the below configuration.
Configure Jenkins Server
Step 1: Head over to Jenkins Dashboard –> Manage Jenkins –> Manage Plugins.
Step 2: Under the Available tab, search for “Docker” and install the docker cloud plugin and restart Jenkins. Here is the official plugin site. Make sure you install the right plugin as shown below.
Step 3: Once installed, head over to Jenkins Dashboard –> Manage Jenkins –>Configure system.
Step 4: Under “Configure System”, if you scroll down, there will be a section named “cloud” at the last. There you can fill out the docker host parameters for spinning up the slaves.
Note: In Jenkins versions 2.200 or later you will find dedicated cloud configuration under Manage Jenkins –> Manage Nodes and Clouds
Step 5: Under docker, you need to fill out the details as shown in the image below.
Note: Replace “Docker URI” with your docker host IP. For example,
tcp://10.128.0.3:4243You can use the “Test connection” to test if Jenkins is able to connect to the Docker host.
Step 6: Now, from “Docker Agent Template” dropdown, click the “Add Docker template” and fill in the details based on the explanation and the image given below and save the configuration.
- Labels – Identification for the docker host. It will be used in the Job configuration. Here we use
- Name: Name of the docker template. Here we use the same name as label ie, java-docker-slave
- Docker Image –
bibinwilson/jenkins-slave:latestor the image that you created for the slave.
- Remote Filing System Root – Home folder for the user you have created. In our case, it’s /home/jenkins
- Credentials – click add and enter the SSH username and password that you have created for the docker image. Leave the rest of the configuration as shown in the image below and click save. If you are using my Docker image, the user will be
jenkins& password is also
Test Docker Slaves Using FreeStyle Job
Now that you have the slave configurations ready,
- Create a freestyle job, select “
Restrict where this project can be run” option and select the docker host as a slave using the label.
- Add a shell build step which echoes a simple “
If you have done all the configurations right, Jenkins will spin up a container, builds the project and destroys the container once the build is done.
First you will see a pending notification as Jenkins tries to deploy a container on run time and establishes an SSH connection. After a few seconds, your job will start building.
You can check the build logs in your jobs console output as shown below.
- Jenkins will not be able to deploy containers on the host:- Please make sure you have proper connectivity to the docker host on API port.
- Jenkins builds goes in pending state forever:- Make sure you have Docker ports access from Jenkins to docker host.
In this article, I walked you through the process of setting up dynamic slaves using Docker. It can be further customized to fit your specific use cases. Let me know your thoughts in the comment section. Also, don’t forget to share this article 🙂