How To Create Kubernetes Jobs/Cron Jobs – Getting Started Guide

Kubernetes jobs are primarily meant for short-lived and batch workloads. It runs for completion as opposed to other objects like replicasets, replication controllers and DaemonSets which runs continuously. This tutorial explains the process of creating kubernetes jobs and cron jobs along with few tips and tricks.

Jobs run until the tasks specified in the job is completed. Which means, if the pods give exit code 0, the job will exit. Whereas in normal Kubernetes deployments, irrespective of the exit codes, the deployment object will create new pods when it terminates or throws an error to keep the desired stated of the deployment.

Also, during a Job run, if the node hosting the pod fails, the job pod will get automatically rescheduled to another node.

Kubernetes Jobs Use Case

The best use case for Kubernetes jobs are,

1. Batch processing: Let’s say you want to run a batch task one time a day or during a specific schedule. It could be something like reading files from storage or a database and feed it to a service to process the files.
2. Operations/ad-hoc tasks: Let’s say you want to run a script/code which runs a database cleanup activity.

Creating a Kubernetes Job

In this example, we will use an Ubuntu container which runs a for loop and echoes a message based on the argument you pass to the container. An argument is a number which decides how many times the loop should run.

You Might Like: Get 16% Off + $100 Off On Kubernetes Certification & Course Bundle

For example, if you pass 100 as an argument, it will echo the message 100 times and the container will exit.

You can view the Dockerfile and script from here -> kube-job-example Docker configs

Lets get started with job with a simple setup.

1. Create a job.yaml file with our custom docker image with 100 as a command argument. The value 100 will be passed to docker ENTRYPOINT script as an argument.
   apiVersion: batch/v1 kind: Job metadata: name: kubernetes-job-example labels: jobgroup: jobexample spec: template: metadata: name: kubejob labels: jobgroup: jobexample spec: containers: - name: c image: devopscube/kubernetes-job-demo:latest args: ["100"] restartPolicy: OnFailure

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

   apiVersion: batch/v1 kind: Job metadata:   name: kubernetes-job-example   labels:     jobgroup: jobexample spec:   template:     metadata:       name: kubejob       labels:         jobgroup: jobexample     spec:       containers:       - name: c         image: devopscube/kubernetes-job-demo:latest         args: ["100"]       restartPolicy: OnFailure

2. Let’s create a job using kubectl with the job.yaml file.
   kubectl apply -f job.yaml

   1	kubectl apply -f job.yaml

3. Check the status on job using kubectl.
   kubectl get jobs

   1	kubectl get jobs

4. You can get the list of pods using kubectl.
   kubectl get po

   1	kubectl get po

5. You can get the job pod logs using kubectl.
   kubectl logs kubernetes-job-example-bc7s9 -f

   1	kubectl logs kubernetes-job-example-bc7s9 -f

Multiple Job Pods and Parallelism

When a job is deployed you can make it run on multiple pods with parallelism.

For example, if you want to run 6 pods and run 2 pods in parallel, you need to add the following two parameters to your job manifest.

Here is the manifest file with those parameters.

Generate Random Job Name

You cannot have a sigle job manifest file and create multiple jobs from it. Kubernetes will throw an error stating that job with the same name exists.

To circumvent this problem, you can add the generateName name parameter to the metadata.

For example,

In the above example, every time you run the manifest, a job will get created with kube-job- as a prefix followed by a random string.

Creating a Kubernetes CronJob

What if you want to run a batch job on specific schedules, for example, every 2 hours. You can create a Kubernetes cronjob with a cron expression. The job will automatically kick in as per the schedule you mention in the job.

Here is how we specify a cron schedule. You can use the crontab generator to generate your own schedule.

If we were to run our previous job as a cron job every 15 minutes, here is how the manifest looks. Create a file named cron-job.yaml and copy the following manifest.

Lets deploy the cronjob using kubectl.

List the cronjobs

Trigger a CronJob Manually

There are situations where you might want to execute the cronjob in an ad-hoc manner. You can do this by creating a job from an existing cron job.

For example, if you want a cronjob to be triggered manually, here is what we should do.

--from=cronjob/kubernetes-cron-job will copy the cronjob template and creates a job named manual-cron-job

Few Key Parameters

There are few more key parameters you can use with kubernetes jobs/cronjobs based on your needs. Lets have a look at each.

1. failedJobHistoryLimit & successfulJobsHistoryLimit: Deletes the failed and successful job history based on the retention number you provide. This is super useful to trim down all failed entries when you try to list the jobs. For example,
   failedJobHistoryLimit: 5 successfulJobsHistoryLimit: 10

   1 2	failedJobHistoryLimit: 5 successfulJobsHistoryLimit: 10

2. backoffLimit: Total number of retries if your pod fails.