How to Setup AWS ECS Cluster as Build Slave for Jenkins

In our last post, we wrote about setting up docker containers as build slaves. Integrating docker into your build pipeline has lots of advantages. Especially when it comes to ECS cluster based build slave setup, the advantages are even more. Few of them are,

1. In teams where continuous development happens, most the time the slave machines will be idle. By using ECS you can save cost by reducing the jenkins slave machines. The cluster capacity increases when there is a need for more resources.

2. Manage build environments in containers with tagged versions as compared to installing several versions of an application on the same machine.

3. Optimal usage of system resources by spinning up containers only when needed.

You Might LikeLearn Docker Technologies for DevOps and Developers

Configuring ECS Cluster As Build Slave

In this post, we will guide you to setup jenkins slaves on an ECS cluster. In this setup, we have the following.

1. A working jenkins master(2.42-latest) running on a container, set up under an ELB.

Note: You can follow this article for setting up the latest Jenkins server using a container.

2. An ECS cluster with 2 Container instances. You can decide on the container instance capacity based on your project needs.

Note: It is better to have an ASG configured for the instances to scale the ECS cluster on demand.

Prerequisites

1. In ec2 and ELB security groups, you should open the jnlp (50000) port for container slaves to connect to jenkins master.

2. Go to Manage Jenkns –> Configure Global Security, check,Enable Security Option check fixed and enter 50000 as shown in the image below.

Follow the steps given below for configuring jenkins with your existing ECS cluster.

Install Amazon EC2 Container Service Cloud Plugin

1. Go to Manage Jenkins –> Manage Plugins and search for Amazon EC2 Container Service Plugin

2. Install the plugin and restart it.

READ  Docker Networking Tutorial : Connect Containers Across Hosts Using Weave

Configure ECS settings on Jenkins

To integrate ECS with the jekins master, jenkins should have AWS SDK access. You can enable this in two ways.

1. You can add an IAM role with EC2 Container service Full Access to the instance where you have installed the jenkins server.

2. You can add AWS access key and secret key to the Jenkins credentials and use it with the ECS configuration. If you are running jenkins in a container outside ECS, this will be the only available option. If you are running your jenkins server in ECS, then you can assign it a task role having privileges to ECS cluster.

In this tutorial, I am using the access key and secret key stored in Jenkins AWS credentials.

Follow the steps given below for configuring the ECS plugin to integrate the ECS cluster.

1. Go to Manage Jenkins –> Configure System and search for cloud option.

2. Click Add a new cloud dropdown and select Amazon EC2 Container Service Cloud

3. You need to fill up the following details.

Name: User defined name.

Amazon ECS Credentials: If you are using AWS access keys, select the relevant credential. If you using AWS role, leave it empty. The cluster will get listed automatically.

Amazon ECS Region Name: AWS region where you have the cluster.

ECS Cluster: Select your ECS cluster from the drop-down.

An example is shown below.

If you are running jenkins master under an ELB, you need to add the tunnel configuration in the advanced section.

The Tunnel connection through option should have the elb URL followed by the JNLP port as shown below.

4. Next, you need to add a slave template with a docker image which acts as a slave node.

To do this, under ECS slave templates in the cloud configuration, click add and enter the details. In the following example, I have added a java JNLP slave image with label java-ecs-salve. Label is very important, because, we will use the label name in the job to restrict the job to run on containers.

READ  List of Popular Open Source Java Build Tools

After filling out the details, save the configuration.

Test The Configuration

To test the configuration, we will create a Freestyle project and try to run it on ECS.

1. Go to New item and create a freestyle project.

2. Under Restrict where this project can be run type the label name you have given in the slave template as shown in the image below.

3. Under Build select the execute shell option and type an echo statement as shown below.

4. Save and job and click build now

The build will go the pending start once it deploys the container. Once it executes the shell the container will be destroyed from ECS.

If you click the console output, you should get the following output.

Once you tested your configuration, you can create actual jobs by integrating with git and the build steps that you want. You need to configure the docker image based on your requirement.

For java based projects you can use the following cloudbees image as slave templates. It contains most of the java build dependencies.

1. https://hub.docker.com/r/cloudbees/jnlp-slave-with-java-build-tools/~/dockerfile/

For projects other than java, you can use the following cloudbees JNLP image we used in the demo as the base image.

1. https://hub.docker.com/r/jenkinsci/jnlp-slave/

ECS Cluster As Build Slave will save time and cost. If you have your applications of AWS, it is worth giving it a try. You can start with few applications and once you gain confidence, you can move the entire build fleet to ECS container based build pipelines.

Contact us at [email protected] or leave a comment for any help with this implementation.

Free DevOps Resources

Get DevOps news, tutorials and resources in your inbox. A perfect way If you want to get started with devops. Like you, we dont like spam.

5 comments

  1. suryatej yaramada Reply

    Hi, it was the very nice explanation which helps me to configure and it works perfectly. I can run jobs which create Docker containers dynamically and destroys after job completed but the only problem am facing is workspace issue which is unable to access through Jenkins UI

    Error: No workspace

    so please let me know what needs to be done to get the workspace after job is run in docker-executor

    Thanks

    • devopscube Post authorReply

      Hi Surya,

      The docker container gets terminated after the build. That’s the reason you are unable to access the workspace. In this case you might need to look into mounting your docker container to EFS and make it as a workspace. So that the data will persist.

  2. surya Reply

    Hi,
    Thanks for a response can you please post a blog for how to achieve this if possible and also is there another alternative as EFS would be more costly.

    Thanks

  3. surya Reply

    Hi ,
    I able to mount EFS but unable to get the data from the containers as they create dynamically so please let me know how this can be achieved.

    Thanks & Regards
    Surya

  4. Fadwa Messaoudi Reply

    is there a way to change the the CPU and RAM values of the task definition using the jobs , so you don’t do it manually ?

    Thanks,
    Fadwa.

Leave a Reply

Your email address will not be published. Required fields are marked *