This blog is for you if you are looking for the best devops projects for learning and practice. It contains a curated list of real-world devops project scenarios that will help you learn the concepts and well as gain experience with DevOps tools.
As per alliedmarketresearch, the Devops market is projected to reach $57.90 billion by 2030, registering a CAGR of 24.2% from 2021 to 2030. So, organizations need DevOps engineers with hands-on knowledge, and you need to demonstrate those skills
The best way to show your skillsets is by working on real-world devops projects.
All the projects listed in this blog based on real-world learning to gain hands-on experience. It includes the following.
- Project Architecture
- Project Workflow
- Full Infrastructure source code in GitHub
- Project Documentation
All of these DevOps projects are structured to provide practical learning opportunities based on my 10+ years of experience working as a cloud and DevOps engineer.
Note: It is a growing list.
DevOps Projects Github Repository
All the code for the projects listed in this guide is present in the Github DevOps projects repository. It contains all the CLI commands, configuration management codes, IaC codes, and much more. You can clone the repo using the following command.
git clone https://github.com/techiescamp/devops-projectsList of DevOps Projects For Learning
If you are a fresh graduate or an experienced person looking to start your DevOps journey, it is very important to learn the basics. The following set of projects will help you learn some of the core IT fundamentals and DevOps real-world to help you prepare for DevOps engineer interviews.
Note: You dont need to spend any money for the labs for theses projects. You can use vagrant based VMs locally or use $1000+ free cloud credits on AWS, Google Cloud, Digital Ocean, Linode, Vultur etc
List of DevOps Projects
- Project 1: Setup a Static Website Using Nginx
- Project 2: Setup Multiple Static Websites on a Single Server Using Nginx Virtual Hosts
- Project 3: Setup Load Balancing for Static Website Using Nignx
- Project 4: Setup WordPress Website Using LAMP Stack
- Project 5: Setup Service Discovery Using Nginx & Consul
- Project 6: Create an API Based Application
- Project 7: Design a AWS VPC Based on Application Architecture.
- Project 8: Deploy Scalable Java Application on AWS Cloud
- Project 9: One-Way and Mutual SSL/TLS Implementation
- Project 10: Setup Client to Site VPN on AWS
- Project 11: Deploy Prometheus Stack Using Docker Compose
Project 1: Setup a Static Website Using Nginx
This project aims to help you learn to set up a static website using Nginx and configure DNS to make the website publicly accessible.
| Project Tasks | Key Concepts Covered |
|---|---|
| 1. Buy a domain name from a domain Registrar 2. Spin up a Ubuntu server. 3. SSH into the server and install Nginx 4. Download freely available HTML website files. 5. Using SCP, copy the website files to the Nginx website directory. 6. Validate the website using the server IP address. 7. In your DNS account, create an A record and add the Elastic IP. 8. Use the dig command to verify the DNS records. 9. Using DNS verify the website setup. 10. Create a Letsencryp certificate for the DNS and configure it on the Nginx server. 11. Validate the website SSL using the OpenSSL utility. | 1. DNS 2. Linux 3. Webserver 4. Nginx 5. Dig command 5. SSL (Letsencrypt) 6. OpenSSL command. |
Optionally you can try using a self-signed SSL certificate to understand the difference between self-signed and trusted SSL certificates.
Project 2: Setup Multiple Static Websites on a Single Server Using Nginx Virtual Hosts
In this project, you will learn the concept of subdomains and hosting multiple websites on a single server using Nginx Virtual Host configuration.
| Project Tasks | Key Concepts Covered |
|---|---|
| 1. Create two subdomains 2. Install and configure Nignx on a server 3. Create two website directories with two different website templates. 4. Configure the Virtual host to point two subdomains to two different website directories. 5. Add the IP of the server as A record to the two subdomains. 6. Validate the setup accessing the subdomains. 7. Create a wildcard Letsencrypt SSL certificate for the root Domain. 8. Configure wildcard SSL on Nginx for two websites. 9. Validate the subdomain websites’ SSL using OpenSSL utility. | 1. Subdomains 2. Nginx Virtual Hosts routing. 3. Wildcard SSL (Letsencrypt) |
Project 3: Setup Load Balancing for Static Website Using Nignx
This project aims to learn Layer 7 load balancing and load balancing algorithms using Nginx as a Load Balancer.
| Project Tasks | key Concepts Covered |
|---|---|
| 1. Deploy three servers 2. Set up static websites on two servers using Nginx. Make a small change in the index.html file of one of the websites to differentiate between two servers. 3. Set up Nginx on the third server. It will act as a load balancer. 4. Configure Nginx to load and balance traffic between two static websites. 5. Add the Nginx Load balancer IP to the DNS A record. 6. Try accessing the website. Every time you reload the website you should see a different index.html. 7. Try different Nginx load-balancing algorithms and options. 8. Understand L7 load balancing. | 1. Load balancing 2. Load balancing algorithms 4. L7 Load balancing |
Project 4: Setup WordPress Website Using LAMP Stack
The aim of this project is to learn about Dynamic websites that use Databases. Here you will learn to work with a Relational Database, apache web server, and PHP language.
| Project Tasks | Key Concepts Covered |
|---|---|
| 1. Deploy a Ubuntu server 2. Set up the LAMP stack on the server. 3. Configure WordPress Application 4. Map the IP address to the DNS A record. 5. Access the DNS and validate the WordPress website setup. | 1. Basic Database setup and administration. 2. Setting up open-source PHP application. |
DevOps basics can be learned by setting up a simple WordPress blog for yourself as shown in the following architecture.
You can gain practical experience with the following:
- DNS records: A, CNAME, MX, TXT, and more.
- Content Delivery Network: Cloudflare.
- Web Server Configuration: Apache.
- Reverse Proxy Configuration: Nginx.
- Application configurations: WordPress (PHP).
- Database Management: MySQL.
- SMTP Settings: for email delivery.
- Server/Data Backup: WordPress Backup, MySQL Backup.
- Custom Domain Email: Zoho Mail.
Here’s what it will cost you:
A Domain Name – you can get it for very cheap from Godaddy (~$1)
Everything else is free. You can then use the blog to publish your learnings.
Project 5: Setup Service Discovery Using Nginx & Consul
Service discovery is an important concept every DevOps Engineer should learn. It is the concept of discovering service information in real time. It involves a centralized service registry that maintains information about available services
Service discovery is of two types
- ✅ 𝗖𝗹𝗶𝗲𝗻𝘁 𝗦𝗶𝗱𝗲 𝗦𝗲𝗿𝘃𝗶𝗰𝗲 𝗗𝗶𝘀𝗰𝗼𝘃𝗲𝗿𝘆: In this model, the request goes to a service registry to get the information available for backend services. For example, an application connects to a service registry to get information about the database endpoint.
- ✅ 𝗦𝗲𝗿𝘃𝗲𝗿 𝗦𝗶𝗱𝗲 𝗦𝗲𝗿𝘃𝗶𝗰𝗲 𝗗𝗶𝘀𝗰𝗼𝘃𝗲𝗿𝘆: In this model, the request goes to a load balancer and the load balancer uses the service registry to get the information of the backend servers. The example is shown in the image.
If you implement this project, it will serve as a solid foundation for all the service discovery-based implementations on both VM and container-based implementations.
To understand service discovery practically, here is what you can do.
- Setup Hashicorp Consul (Service registry)
- Setup Nginx load balancer with consul agent to query the consul server (Service registry)
- Configure the nginx.conf as a consul template (Go template) to retrieve backend IPs/Configs in real-time from the service registry.
- Set up backend servers for the load balancer and update the IPs to the service registry.
- Validate the setup by checking if the load balancer is able to serve the traffic by reading backend information from the service registry.
- If you change/update any backend info on the service registry, the consul agent running on the load balancer identifies it and updates the info in the nginx.conf file.
𝗡𝗼𝘁𝗲: This is just a short and practical workflow to understand service discovery. In real projects, more automation is involved in the registry update and retrieval process.
Project 6: Create an API Based Application
Programming is a requirement for DevOps engineers now. Be it for implementations or interviews, it is a must-have knowledge. One way to develop an interest in learning programming is by developing sample web applications. During each module completion, you will have a sense of satisfaction, and your confidence level to learn programming will improve.
My suggestion would be to develop an API-based application using Python Flask or FastAPI to practically understand REST APIs, database connection configuration, and a little bit of UI (HTML & Javascript)
Project 7: Design a AWS VPC Based on Application Architecture.
Every DevOps engineer should have a good understanding of cloud networking. In most organizations, the VPC is managed by the central network team. However, DevOps engineers need to work with those teams to create the required networks for projects by providing all the network details.
The best way to learn cloud networking is by designing and implementing it practically.
In this project you will use a sample application infrastructure architecture to design a AWS VPC with standard best practices that includes most of the AWS VPC concepts.
| Project Tasks | Key Concepts Covered |
|---|---|
| 1. Understand VPC Requirements 2. VPC Network Design 3. Subnet Design 4. VPC Documentation 5. Route Table Design 6. AWS VPC Topology 7. Network ACLs 8. VPC Endpoints | Creating VPC design based on application infrastructure architecture |
Project Documentation: Design AWS VPC
Project 8: Deploy Scalable Java Application on AWS Cloud
In this project, you will learn to build and deploy a Java application on the AWS platform.
Project Code: Scalable Java Application on AWS Code
| Project Tasks | Key Concepts Covered |
|---|---|
| 1. Build a Java application 2. Use Packer & Ansible to build the AMI With application code, cloudwatch agent for application logging, prometheus JMX exporter for application metrics, and consul agent to register service information. 3. Use Terraform to provision MySQL RDS instance and to store the MySQL username and password in the AWS secrets manager 4. Provision Load Balancer 5. Provision Autoscaling Group with Launch template that uses AMI built by packer and attaches it to Loadbalancer. 6. In the ec2 user data use the boto3 script to retrieve the RDS password in realtime that could be used by the application. 7. Verify the application by accessing it using the Load Balancer endpoint. 8. Verify application logs in Cloudwatch 9. Verify application metrics in Prometheus. 10. Test application autoscaling 11. Creating monitoring dashboards on Cloudwatch. | 1. Java application build. 2. Immutable infrastructure model for java application 3. AWS cloudwatch custom logging 4. AWS Autoscaling 5. AWS Loadbalancer 6. AWS secrets manager. 7. IaC using Packer, Terraform and Ansible. 8. Consul for service discovery 9. Prometheus for application monitoring |
Here is the high-level application architecture.
Project 9: One-Way and Mutual SSL/TLS Implementation
The Idea of this project is to learn the concepts of oneway and mutual SSL/TLS implementations.
For practical learning, you can start with One Way SSL
Only the server must have a digital certificate in One-way SSL to establish a secure connection.
- Deploy an Nginx website over a valid domain name.
- Next, you can set up Let’s Encrypt SSL, a certificate authority that provides free SSL/TLS certificates.
- You can also set up a wildcard domain and configure Let’s Encrypt to understand wildcard DNS and SSL.
Another concept to learn is Mutual TLS, which is common in most distributed systems. For example, Kubernetes components interact via mTLS.
Also, you will encounter this concept in server mesh implementations.
mTLS (Mutual TLS) is a security protocol that allows both client and server to authenticate each other using certificates. To learn Mutual TLS, you can generate client and server certificates using OpenSSL.
Next, you can create a simple Python Flask API that requires SSL and a client program that makes a request to the Flask API using its certificates.
Finally, you can verify that the request succeeds and that the client and server exchange SSL/TLS certificates.
Project 10: Setup Client to Site VPN on AWS
Most devops engineers work in cloud environments. One of the key requirements in such environments is VPN connectivity to access the cloud resources securely.
Organizations typically use site-to-site VPN connectivity. However, for learning purposes, we can set up a client-to-site VPN on AWS and understand all the key concepts involved.
We have created a mini project to learn the client-to-site VPN setup.
Project Documentation: AWS Client VPN Endpoint Setup
Project 11: Deploy Prometheus Stack Using Docker Compose
The idea of this project its to put your Docker knowledge in to a hands on project.
The project aims at implementing the following concepts.
- Infrastructure as Code using Terraform
- Multi-Container Deployments using Docker Compose
- Monitoring & Alerting using Prometheus Stack
You will learn the following from the project.
- Provision an ec2 server using well-structured Terraform code.
- Deploy prometheus, Alert Manager, Node Exporter, and Grafana as Docker containers using Docker Compose.
- The project also makes use of Shell script and Makefile.
Project Documentation: Setup Prometheus Stack & Grafana Using Docker
Upcoming List
Details and breakdowns for the following projects will be added soon.
- Complete Docker Image Build Pipeline
- Implement Forward Proxy on Cloud using Squid Proxy.
- Secret Management With Hashicorp Vault
- Terraform With AWS
- Kubernetes the Hard Way on AWS
- End-to-end application Deployment on Kubernetes
- GtiOps With ArgoCD on Kubernetes
- Entire CI/CD setup for application deployments on Kubernetes
Need for Devops Real World Projects
One of the problems faced by aspiring devops engineers is a lack of hands-on knowledge doing DevOps projects. Often, it becomes a big hurdle in getting a DevOps engineer role or moving into a different domain in DevOps itself. You can overcome this issue by working on real-time projects.
Devops real-time projects help you gain experience with DevOps tools, infrastructure design, and troubleshooting. Moreover while working on projects, you will be able to apply and gain an understanding of Linux, scripting, and other fundamental devops concepts. This way you can confidently attend DevOps interviews.
I have segregated the projects into different categories in terms of beginner, IAC, cloud, and containers. Based on your experience level and understanding of the tool you can pick the projects.
Conclusion
The best way to learn DevOps is by doing practical projects. The idea is to understand the core fundamentals of each implementation. Even though the implementation could differ from organization to organization, the best underlying concepts, workflows and best practices remain almost the same.
In DevOps interviews, companies expect real-time project experience with tools. However, if you have strong knowledge of the tools, even without project experience, they might consider you.
Also, this blog will be constantly updated with more project workflows for real-time learning and production scenarios.
7 comments
This is great Bibin. Thanks for putting this content. I looked at the github from the given link however could not find the first project (Setup a Static Website Using Nginx). Will you please help me to find this one?
Hi vimal I too didnt find the first , Bibin can you help with this
We will publish it soon. All documented projects are available in the Gitrepo
Hi,
Is there any end to end project or any application where we can the practical application of devops tools
Thank you, Bibin, for this content.
thank you very much!
keep up the great content
Welcome David:)