> ## Content Index > Fetch the complete content index at: https://devopscube.com/llms.txt > Use this file to discover other available public pages before exploring further. # How to Build a Kubernetes AI Agent with LangGraph & AWS Bedrock - URL: https://devopscube.com/build-kubernetes-ai-agent/ - Published: 2026-10-27T14:36:57.000Z - Updated: 2026-10-27T15:02:52.000Z - Author: Bibin Wilson - Tags: #syntax-highlight, AI/ML, MLOPS, #kubernetes, #blog In the [AI Agent architecture](https://devopscube.com/ai-agent-architecture/) blog, we covered the core concepts of AI agents. In this guide, let's use those concepts to build a Kubernetes AI Agent from scratch. The agent we will build will investigate a Kubernetes issue, use tools to gather information, reason about what it finds, propose a fix, wait for our approval, and verify whether the fix worked. Also, along the way, you will see how the **agent loop, tools, state, and human approval** concepts fit into a real implementation. At the end of this guide, you will have the following: - Understanding of an Agent code structure - Where the AI Agent concepts fit into a real implementation - A working Kubernetes troubleshooting agent - And a clear idea of how the agent works in Kubernetes troubleshooting ## What Are We Building? We are building a [Kubernetes](https://devopscube.com/kubernetes-architecture-explained/) agent that troubleshoots the cluster using plain-English requests. It investigates the live cluster using resource details, events, and logs, identifies the root cause, and proposes a fix. Before making a change, the agent **waits for your approval.** You can approve, reject, or provide a custom fix instead. Once approved, it applies the change using a relevant function tool, then rechecks the cluster to verify the issue is resolved. If the problem remains, it proposes another fix. Here is the agent architecture. ![Architecture diagram of an AI-powered Kubernetes troubleshooting agent built with FastAPI, LangGraph, LangChain, and AWS Bedrock](https://storage.ghost.io/c/5f/2f/5f2f4d20-2abf-4534-8d40-7aa233aedd43/content/images/2026/09/image-105.png) Before we start building, I want to make one thing clear. The **goal here** isn’t to say every [DevOps engineer](https://devopscube.com/become-devops-engineer/) should build their own AI agents. In real project environments, you might use an existing agent and connect it to your internal systems, or build one for a specific use case. Either way, from an infrastructure perspective, you should understand: - **Tool access:** How agents interact with Kubernetes, cloud APIs, monitoring systems, GitHub, and other tools. - **Context:** How agents use logs, metrics, alerts, documentation, and runbooks. - **Security:** How identity, RBAC, secrets, and permissions control what an agent can do. - **Observability:** How to trace LLM calls, tool calls, failures, latency, and agent actions. We will cover these concepts while building the agent. But first, let’s look at how infrastructure teams use AI today. ## **How Are Infrastructure Teams Using AI Today?** Before we build our agent, here is some interesting data. Pulumi's [2026 State of Agentic Infrastructure](https://www.pulumi.com/state-of-agentic-infrastructure?ref=devopscube.com) survey of 510 engineers found: - 24% reported using in-house agents for infrastructure work. - 81% allow agents to make production infrastructure changes - 62% require human approval. - Only 19% allow agents to make those changes autonomously. The following image shows where engineers use AI agents in infra. ![Survey report showing how Infrastructure Teams Using AI agents](https://storage.ghost.io/c/5f/2f/5f2f4d20-2abf-4534-8d40-7aa233aedd43/content/images/2026/10/image-131.png) img source: [pulumi.com report](https://www.pulumi.com/state-of-agentic-infrastructure/?ref=devopscube.com) Now, let's put everything we've learned so far into practice with a project. ## Setup Prerequisites Below are the prerequisites. - [AWS EKS Cluster](https://devopscube.com/create-aws-eks-cluster-eksctl/) with pod-identity-agent addon enabled - [kubectl](https://devopscube.com/kubectl-set-context/) - [Docker](https://devopscube.com/how-to-install-and-configure-docker/) - Two Docker repositories - [AWS CLI](https://devopscube.com/install-configure-aws-cli-linux/) with access to [AWS IAM](https://devopscube.com/aws-iam-role-instance-profile/) and EKS - [Kustomize](https://devopscube.com/kustomize-tutorial/) ## **Agent Project Structure** We have pushed all code files to our GitHub repository. Fork and clone the repository to your local machine, then navigate into the **`kubernetes-ai-projects/ai-agent`** folder. ```bash git clone https://github.com/techiescamp/kubernetes-ai-projects.git cd kubernetes-ai-projects/ai-agent ``` Inside the **`ai-agent`** folder, these are the key files of the agent backend and frontend interface. The following image explains what the code files do. ![kuberntes ai agent source code directory structure](https://storage.ghost.io/c/5f/2f/5f2f4d20-2abf-4534-8d40-7aa233aedd43/content/images/2026/10/image-85.png) In this, the **`agent-backend`** folder contains the agent code. Let's get an overview of what it does. ### Agent Backend This is the structure of the backend source code folder. ```bash agent-backend/app/ ├── main.py ├── agent/ ├── tools/ └── infra/ ``` The **`backend/app`** folder contains 1 main Python file and three folders as shown above; let's go through it. #### main.py The backend is built with [FastAPI](https://fastapi.tiangolo.com/?ref=devopscube.com), and this file handles the HTTP endpoints. Below are the key endpoints available for the agent. | API Endpoint | Description | | ------------------ | ------------------------------------------------------------------- | | /api/query | Starts a session. | | /api/decision | Handles approval or rejection requests. | | /api/select-issues | Lets you select which issues to fix when multiple issues are found. | | /api/guidance | Handles custom instructions. | | /api/retry | Retries a failed fix. | | /healthz | Checks the health of the agent. | | /metrics | Exposes application metrics. | #### agent/ This folder contains the agent logic of how it should work. ```bash agent/ ├── state.py ├── graph.py ├── nodes.py └── parsing.py ``` What each file does: | File | Description | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | nodes.py | Contains the actual logic for each step in the workflow. | | state.py | Defines the shared fields used by all steps, such as issues, retry\_context, and goal\_achieved. LangGraph uses this state to pass information between steps. | | graph.py | Defines how the steps in nodes.py are connected and determines their execution order. | | parsing.py | Extracts specific fields, such as detected issues and routing decisions, from the model's response. | #### tools/ This folder contains the operations logic the agent can do on the Kubernetes cluster. ```bash tools/ ├── kube.py ├── read.py ├── write.py └── guards.py ``` What each file does: | File | Description | | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | kube.py | Manages the connection between the agent and the cluster using the pods own service account token and CA certificate, and cleans up raw K8s error messages into something the ML model can actually use. | | read.py | Contains tools that perform read operations, such as listing resources, reading logs, and checking events | | write.py | Contains tools that can do actions like creating, deleting, and updating. | | guards.py | This is the guardrail logic for the agent, which blocks certain write operations like modifying the agent’s own RBAC or healthy system workloads. | Every tool is a plain Python function with a **`@tool`** decorator. The decorator reads the function's argument names, types, and docstring and turns them into a JSON schema. It sends that schema to the LLM, so it knows the function exists and when to call it. 💡 The docstring are tool description shown to the ML model, so it must clearly explain what the tool does The LLM never runs the function. When it decides to use it, it outputs a structured request with the tool name and arguments. Your code reads that and calls the actual Python function, and gives the result back to the model as text. The tools do the actual work on the cluster, and the LLM decides what to ask for. The current agent has **24 read tools and 11 write tools** configured. #### infra/ This folder contains files that give the agent additional capabilities, like [l**ogging**](https://devopscube.com/kubernetes-logging-tutorial/)**, storage, and a token usage tracker.** ```bash infra/ ├── bedrock.py ├── checkpointer.py ├── conversation_store.py ├── logging_config.py ├── metrics.py └── usage_tracker.py ``` What each file does: | File | Description | | ---------------------- | ------------------------------------------------------------------------------------------------------------------ | | bedrock.py | Uses boto3 to connect with the model on AWS Bedrock | | checkpointer.py | Contains configuration to connect with PostgreSQL and also, if PostgreSQL is not configured, saves data in memory. | | conversation\_store.py | Saves session history in PostgreSQL. | | logging\_config.py | Exposes application logs on STDOUT. | | metrics.py | Exposes application metrics like session duration, tools called, token usage in /metrics. | | usage\_tracker.py | Tracks token usage and shows the usage on the UI | The agent can use **two separate ML models**, one for diagnosis and verification, and another for fixing. This lets us select an ML model suited for analysis and fixing. We can also use the same model for both if needed. **LangGraph** controls the workflow order and the tools used at each stage. Diagnosis, proposal, and verification tools only have read-only access, and write tools are used only after the user approves the fix plan. ### Agent Interface (Web UI) The agent also has a chat interface that allows users to send requests in plain English and view troubleshooting results. The interface is built with React and [Next.js](https://nextjs.org/docs?ref=devopscube.com). React provides the interactive user interface, while Next.js adds routing, build tooling, and server-side request handling. The application runs on Node.js because the Next.js server executes **`route.ts`**, a Route Handler that acts as an API proxy. It receives **`/api/*`** requests from the UI and forwards them to the FastAPI backend. This is the structure of the interface's source code folder. ```bash agent-interface/app/ ├── page.tsx ├── layout.tsx ├── globals.css └── /api/[...path]/route.ts ``` The files contain: | File | Description | | ------------------------- | ------------------------------------------------------------------- | | page.tsx | Main chat page code | | layout.tsx | Page layout code | | globals.css | Application design code | | /api/\[...path\]/route.ts | Receives /api/\* requests and forwards them to the FastAPI backend. | An agent doesn't always need a chat interface. It can also run through an API, command-line interface, scheduled job, or automation workflow. Now, you got a good idea about an agent, lets build and deploy one. ## Containerize the Agent Code We have built the agent using Docker and pushed the images to our DockerHub registry. 💡 You can skip this step, if you are going to use our image. If you are going to build your own image, build two images: one for the agent's frontend and one for the backend. Run the following commands from the **ai-agent** folder to build and push the images to your registry. ```bash export REGISTRY= export TAG=v1.0.0 docker build -t $REGISTRY/ai-agent-backend:$TAG agent-backend/ docker build -t $REGISTRY/ai-agent-frontend:$TAG agent-interface/ docker push $REGISTRY/ai-agent-backend:$TAG docker push $REGISTRY/ai-agent-frontend:$TAG ``` Now, lets move on to the deployment part. ## **Hands-On: Deploy the Agent on EKS** In this section, we will set up AWS Bedrock access to the agent using Pod Identity and deploy the agent in the cluster using Kustomize. The diagram below gives you an overview of what we are going to set up. ![Ai Agent Architecture on AWS EKS](https://storage.ghost.io/c/5f/2f/5f2f4d20-2abf-4534-8d40-7aa233aedd43/content/images/2026/10/image-3.png) Let's get started. ### Step 1: Choose ML Models from AWS Bedrock For this agent, we used **Amazon's Nova Pro model** for both diagnosis and remediation. We chose this model because it needs to support calling tools and checking the logs, events, and tool results before making a decision. For this, the model needs to handle long inputs, and the **Nova Pro model is good at analysis, question answering**, text generation, and summarization. ![Amazon's Nove Pro models endpoint](https://storage.ghost.io/c/5f/2f/5f2f4d20-2abf-4534-8d40-7aa233aedd43/content/images/2026/10/Screenshot-2026-10-01-at-1.05.43-PM.png) We are not saying this is the best model for a Kubernetes agent; we are saying this is one of the suitable models. 💡 To use the model, we don’t have to deploy or activate it in Bedrock. The model is serverless; we just need its ID and permission to call it. Amazon's Nova Pro model costs **$0.80 per million input tokens** and **$3.20 per million output tokens** in the Oregon region. Check the [Amazon pricing page](https://aws.amazon.com/bedrock/pricing/?ref=devopscube.com) for current model costs. 💡 Also, use the region prefix with the model ID. For example, when using the model from the region ****us-west-2** and the model ID is ****amazon.nova-pro-v1:0,** use it as ****us.amazon.nova-pro-v1:0**. ### **Step 2: Set Up AWS Permissions with Pod Identity** The agent calls Bedrock for the ML model, so it needs an IAM role. We are going to use **EKS Pod Identity** to attach a role to the agent. For that, we have created a custom script to create a role and attach it to the service account. Move into the **`k8s/scripts`** folder. ```bash cd k8s/scripts ``` Then, open the **`setup-pod-identity.sh`** script and update your **cluster name**. After updating the cluster name, run the following commands to run the script. ```bash chmod +x setup-pod-identity.sh ./setup-pod-identity.sh create ``` ### Step 3: Configure the Kustomization File Before deploying, we need to modify the **`kustomization.yaml`** in the **`k8s`** folder. For that, move out of the scripts folder into the **`k8s`** folder. Then, open the **`kustomization.yaml`** and update the following highlighted configs in the image. ![](https://storage.ghost.io/c/5f/2f/5f2f4d20-2abf-4534-8d40-7aa233aedd43/content/images/2026/10/image-2.png) - Docker image, if you are using your own image - Update the region, model name, and their token cost under the **configMapGenerator** block if you are using a different model. - The model we are using is **Amazon's `amazon.nova-pro-v1:0`** for both diagnosing and fixing the issue. - Then update the PostgreSQL password under the **secretGenerator** block. ⚠️ If you are using a Kubernetes cluster other than EKS, uncomment the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` and update it with AWS credentials. The agent currently only supports models from AWS bedrock. ### Step 4: Deploying the Agent Now that everything is set, let's deploy the agent in the cluster. 💡 Make sure your cluster have a default storageclass Run the following command inside the k8s folder. ```bash kubectl apply -k . ``` This uses the **`kustomization.yaml`** to apply all the manifests. Once applied, use the following command to verify that the frontend, backend, and PostgreSQL pods are running. ```bash $ kubectl get po -n ai-agent NAME READY STATUS RESTARTS AGE ai-agent-backend-5ccd887dbb-p84pr 1/1 Running 0 1m ai-agent-frontend-79cc74499f-x76s4 1/1 Running 0 1m ai-agent-postgres-0 1/1 Running 0 1m ``` ### Step 5: Access the Agent Web UI To access the UI, we will port-forward the frontend service and access it through that port. Use the following command to port-forward the service. ```bash kubectl -n ai-agent port-forward svc/ai-agent-frontend 3000:3000 ``` This will expose the agent's frontend on port **`3000`**. You can access the UI at URL [**http://localhost:3000**](http://localhost:3000/?ref=devopscube.com) as shown below. ![Kubernetes Ai Agent Web UI](https://storage.ghost.io/c/5f/2f/5f2f4d20-2abf-4534-8d40-7aa233aedd43/content/images/2026/09/image-70.png) The agent is now up and running. In the next section, we will test the agent with a troubleshooting task. ## Testing the Agent To test the agent, we will **simulate an issue** by deploying a pod with a nodeSelector that matches no nodes and **ask the agent to troubleshoot it.** First, deploy the pod using the following command. ```bash cat <