> ## Content Index > Fetch the complete content index at: https://devopscube.com/llms.txt > Use this file to discover other available public pages before exploring further. # GPU Sharing in Kubernetes: Step-by-Step Guide - URL: https://devopscube.com/gpu-sharing-in-kubernetes/ - Published: 2026-10-24T13:49:50.000Z - Updated: 2026-10-24T14:35:09.000Z - Author: Aswin Vijayan - Tags: #blog, AI/ML, MLOPS, #kubernetes, #syntax-highlight When a workload requests a GPU in [Kubernetes](https://devopscube.com/kubernetes-tutorials-beginners/), by default, it allocates an entire GPU, even if the workload uses only a small portion of the GPU. GPU sharing solves this by allowing workloads to share the same GPU. In this guide, we will configure and test the three NVIDIA GPU sharing strategies in Kubernetes, such as **Time-Slicing, MPS, and MIG**. We will use the [NVIDIA GPU Operator](https://devopscube.com/setup-gpu-operator-kubernetes/) and its custom Helm values to configure each strategy. Also, we used the **NVIDIA H100 GPU** because we are configuring and testing NVIDIA MIG as well, which needs a [MIG supported GPU type](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/supported-gpus.html?ref=devopscube.com), and the H100 supports MIG. If you are using a different GPU type. make sure it supports MIG. Lets get started. ## Setup Prerequisites Below are the prerequisites for this setup. - [Kubernetes Cluster](https://devopscube.com/setup-kubernetes-cluster-kubeadm/) with an NVIDIA GPU node - [kubectl](https://devopscube.com/kubectl-set-context/) - [Helm](https://devopscube.com/create-helm-chart/) 💡 You can try this on any Kubernetes cluster which has atleast one NVIDIA GPU node avaibale ## Clone Repository We have pushed every Helm values file and test file for each sharing strategy into our GitHub repository. Fork and clone the repository locally. Then move into the **`mlops-for-devops/platform-tools/gpu-sharing`** directory. ```bash git clone https://github.com/techiescamp/mlops-for-devops.git cd mlops-for-devops/phase-3-gpu-infra/gpu-sharing ``` There you can find the following directory structure. ```bash gpu-sharing ├── mig │ ├── mix │ │ ├── mig-heavy.yaml │ │ ├── mig-light.yaml │ │ └── values.yaml │ └── single │ ├── mig-single.yaml │ └── values.yaml ├── mps │ ├── mps.yaml │ └── values.yaml └── slicing ├── slicing.yaml └── values.yaml ``` Lets look into each strategy in detail with hands-on example. ## How to Share GPUs in Kubernetes Lets start the hands-on section. Here, we will configure and test each sharing strategy one by one. ### Add NVIDIA GPU Operator Helm Chart Since we are going to use the NVIDIA GPU Operator Helm Chart, add its repository first if it is not already added. Run the following command to add it. ```bash helm repo add nvidia https://helm.ngc.nvidia.com/nvidia \ && helm repo update ``` We will be using custom Helm values of the NVIDIA GPU Operator to configure and use each sharing strategy one by one and deploy an example workload on it. ### Configure Time-Slicing Strategy Lets start with slicing. In this section, we will use a custom Helm values file to enable time-slicing GPU sharing and deploy a test deployment to test it. We will use the GPU Operator Helm chart to do the configurations. Assuming you are in the **`mlops-for-devops/phase-3-gpu-infra/gpu-sharing`** directory, move into the **`slicing`** directory using the following command. ```bash cd slicing ``` There you can find the values file **`values.yaml`** and a deployment file **`slicing.yaml`** which we are going to use in this section. In the values file, it is set to **`replicas: 4`**, so the GPU will be advertised as 4 GPUs to Kubernetes, and it can schedule 4 workloads to the GPU at the same time. Now, run the following Helm install command inside the **`slicing`** folder. ```bash helm install --wait gpu-operator \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ -f values.yaml \ --version=v26.7.1 ``` Once the installation is completed, run the following command with your GPU node's name. ```bash kubectl describe no | grep -A10 "Allocatable:" ``` You will get a similar output showing that there are 4 GPUs. ```bash Allocatable: cpu: 23840m ephemeral-storage: 684666794899 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 235610848Ki nvidia.com/gpu: 4 pods: 110 ``` Also, there will be labels added to the GPU node based on the sharing strategy we use. Run the following command to check if the node is labeled with **`time-slicing`** label. ```bash kubectl get no --show-labels | grep time-slicing ``` You can see the following label in the output. ```bash nvidia.com/gpu.sharing-strategy=time-slicing ``` #### Testing Time Slicing Strategy For testing the time slicing strategy, we will use the manifest file **`slicing.yaml`** inside the **`slicing`** folder. The manifest will create a deployment with 4 replicas that will use the same GPU. Run the following command to apply it. ```bash kubectl apply -f slicing.yaml ``` Once applied, run the following command to check if the pods are up and running. ```bash $ kubectl get po NAME READY STATUS RESTARTS AGE timeslicing-demo-6f974bdf84-hlpsp 1/1 Running 0 8s timeslicing-demo-6f974bdf84-px457 1/1 Running 0 8s timeslicing-demo-6f974bdf84-wsmwz 1/1 Running 0 8s timeslicing-demo-6f974bdf84-zg9mj 1/1 Running 0 8s ``` Always keep in mind that, while using the Time Slicing Strategy, all workloads share the same GPU without any isolation. To check it, you use the following command. ```bash for pod in $(kubectl get pods -l app=timeslicing-demo -o name); do echo "--- $pod ---" kubectl exec "$pod" -- \ nvidia-smi --query-gpu=uuid --format=csv,noheader done ``` You can see that all the pods use the same GPU. ```bash --- pod/timeslicing-demo-6f974bdf84-4qccg --- GPU-808c2abc-6e32-babd-e395-27e1ce85be2e --- pod/timeslicing-demo-6f974bdf84-lkwwl --- GPU-808c2abc-6e32-babd-e395-27e1ce85be2e --- pod/timeslicing-demo-6f974bdf84-x7ksx --- GPU-808c2abc-6e32-babd-e395-27e1ce85be2e --- pod/timeslicing-demo-6f974bdf84-x9jkd --- GPU-808c2abc-6e32-babd-e395-27e1ce85be2e ``` Before moving to the next strategy, remove the deployment we used for testing time-slicing using the following command. ```bash kubectl delete deployment timeslicing-demo ``` The pods deployed on GPU might take more time to terminate than the pods deployed on CPU. Because GPU workloads need time to clean up VRAM, CUDA, and workload memory from the GPU. ### Configuring MPS Strategy The second strategy we are going to configure is **Multi-Process Service(MPS)**. For that, move into the **`mps`** folder using the following command. ```bash cd ../mps ``` There you can find two files, Helm values file **`values.yaml`** and a deployment file **`mps.yaml`**. Since we will be using the same GPU Operator Helm chart, we will be using the Helm upgrade command to change the strategy to MPS. ```bash helm upgrade --install --wait gpu-operator \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ -f values.yaml \ --version=v26.7.1 ``` Once applied, wait for the pods to restart. For MPS, a new controller pod will be created to manage the MPS strategy. ```bash $ kubectl get po -n gpu-operator | grep mps nvidia-device-plugin-mps-control-daemon-j58wn 2/2 Running 0 2m32s ``` Same as the previous strategy, you will see the following labels on the GPU nodes. ```bash nvidia.com/gpu.sharing-strategy=mps nvidia.com/mps.capable=true ``` #### Testing MPS Strategy For testing MPS, apply the manifest **`mps.yaml`** inside the MPS folder. We will use the same deployment with 4 replicas as we did for the time-slicing strategy. Run the following command to apply the manifest. ```bash kubectl apply -f mps.yaml ``` You can see the following pods running in the default namespace. ```bash $ kubectl get po NAME READY STATUS RESTARTS AGE mps-demo-6f674bdf4-hlssp 1/1 Running 0 6s mps-demo-6f674bdf4-pq457 1/1 Running 0 6s mps-demo-6f674bdf4-wstwz 1/1 Running 0 6s mps-demo-6f674bdf4-zu9mj 1/1 Running 0 6s ``` Since MPS is for equal resource allocation for each replica, we will use the following command to check how much of the GPU each pod is using. ⚠️ Update on of your pods name in the following command ```bash kubectl exec -- \ sh -c ' echo get_default_active_thread_percentage | nvidia-cuda-mps-control ' ``` You will get output as: ```bash 25.0 ``` This means a single pod is limited by MPS to use 25% of the total GPU resource. Because we configured 4 replicas in the MPS configuration, it allocates the GPU resource in 4 equal parts. Even if you reduce the deployment's replica count from 4 to 3, the percentage will be the same because we changed the deployment replica count, not the GPU replica count. If you reduce the GPU replicas from 4 to 3, the usage will be approximately 33% for each replica. Now, remove the deployment before moving to MIG. ```bash kubectl delete deployment mps-demo ``` ### Configuring MIG Strategy Now, lets configure the MIG strategy. Unlike the previous two strategies, MIG slices the GPU at the hardware level instead of just sharing. MIG has two strategy types, single and mixed. ### MIG Single Strategy First, move into the **`mig/single`** folder. ```bash cd ../mig/single ``` There you can find a values file **`values.yaml`** and the deployment file **`mig-single.yaml`**. Then, change to MIG Single strategy using the following Helm upgrade command. 💡 The GPU instance type we are using is **`H100`** so **`1g.10gb`** profile will support for it, if your GPU type is different, check the [supported profiles](https://docs.nvidia.com/datacenter/tesla/mig-user-guide/latest/supported-mig-profiles.html?ref=devopscube.com) and update your supported profile in the values file instead of 1g.10gb. ```bash helm upgrade --install --wait gpu-operator \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ -f values.yaml \ --version=v26.7.1 ``` Once the GPU operator pods are restarted, label the node with the profile name. ⚠️ Update the node name and change the profile name if you are using a different profile ```bash kubectl label node \ nvidia.com/mig.config=all-1g.10gb --overwrite ``` This will restart operator pods, wait until it is done. 💡 We are labeling the nodes because MIG relies on Kubernetes node labels to choose the profile, we need to activate. Then, check the GPU allocation. ```bash kubectl describe no | grep -A10 "Allocatable:" ``` You will get the following output. ```bash Allocatable: cpu: 19850m ephemeral-storage: 684666794899 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 235611936Ki nvidia.com/mig-1g.10gb: 7 pods: 110 ``` #### Testing MIG Single Strategy Lets test the MIG Single strategy by creating a deployment using the **`mig-single.yaml`** manifest. This will create a deployment with 4 replicas that use the MIG profile we configured before. Run the following command to apply it. ```bash kubectl apply -f mig-single.yaml ``` Once applied, you can see the pods running in the default namespace. ```bash $ kubectl get po NAME READY STATUS RESTARTS AGE mig-single-demo-678f69c779-c6g42 1/1 Running 0 2s mig-single-demo-678f69c779-drhhh 1/1 Running 0 2s mig-single-demo-678f69c779-hw5s8 1/1 Running 0 2s mig-single-demo-678f69c779-khm2p 1/1 Running 0 2s ``` And check each pod. ```bash for pod in $(kubectl get pods -l app=mig-single-demo -o name); do echo "--- $pod ---" kubectl exec "$pod" -- nvidia-smi -L done ``` Each pod will have a different MIG UUID as shown below. 💡 MIG UUID is an unique identifier assigned for each MIG slices. ```bash --- pod/mig-single-demo-57658ff6b7-5bkmv --- GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-808c2abc-6e32-babd-e395-27e1ce85be2e) MIG 1g.10gb Device 0: (UUID: MIG-b0abc112-d00d-5ef2-bcc5-78d1c28ac621) --- pod/mig-single-demo-57658ff6b7-82zsd --- GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-808c2abc-6e32-babd-e395-27e1ce85be2e) MIG 1g.10gb Device 0: (UUID: MIG-dbc2c893-6f17-5f5b-90f1-177d8b93a07d) --- pod/mig-single-demo-57658ff6b7-g99kz --- GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-808c2abc-6e32-babd-e395-27e1ce85be2e) MIG 1g.10gb Device 0: (UUID: MIG-c00d8185-f253-59fd-ac09-491a65b20ac9) --- pod/mig-single-demo-57658ff6b7-hjsnk --- GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-808c2abc-6e32-babd-e395-27e1ce85be2e) MIG 1g.10gb Device 0: (UUID: MIG-1d5247a0-b4c5-58b2-9ca5-8e887d6a2441) ``` This means each pod is using isolated MIG slices. Now, delete the deployment before moving to the next section. ```bash kubectl delete deployment mig-single-demo ``` ### MIG Mixed Strategy This is the second type of MIG strategy. With it, we will use different profile types at the same time. Move into the **`mig/mix`** folder using the following command. ```bash cd ../mix ``` There, you can see a Helm values file **`values.yaml`** and two deployment files **`mig-light.yaml`** and **`mig-heavy.yaml`**. First, you need to reset the node label before upgrading. Because the operator needs to clear the existing configuration before changing the strategy. ```bash kubectl label node \ nvidia.com/mig.config=all-disabled --overwrite ``` Now, run the Helm upgrade command to change the strategy to MIG Mix. ⚠️ Change the profile name in the values file, if you are using a different profile ```bash helm upgrade --install --wait gpu-operator \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ -f values.yaml \ --version=v26.7.1 ``` Wait for the operator pods to restart and start running. Then, add the label with the profile name. ```bash kubectl label node \ nvidia.com/mig.config=large-plus-small --overwrite ``` Again, wait for pods to restart and describe the node. ```bash kubectl describe no pool-bj2660mwe-3of18z | grep -A10 "Allocatable:" ``` You can see both profiles are used at the same time. ```bash Allocatable: cpu: 19850m ephemeral-storage: 684666794899 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 235611936Ki nvidia.com/gpu: 0 nvidia.com/mig-1g.10gb: 2 nvidia.com/mig-3g.40gb: 1 pods: 110 ``` #### Testing MIG Mixed Strategy To test the MIG Mixed strategy, we will be using two deployments to use two different profiles. ⚠️ Change the profile names in the deployment files if you are using a different profiles First, apply **`mig-heavy.yaml`** manifest, which creates a deployment that uses **`nvidia.com/mig-3g.40gb`**. ```bash kubectl apply -f mig-heavy.yaml ``` Then, apply **`mig-light.yaml`** manifest, which creates another deployment that uses **`nvidia.com/mig-1g.10gb`**. ```bash kubectl apply -f mig-light.yaml ``` Once both are applied, check the pods are running. ```bash $ kubectl get po NAME READY STATUS RESTARTS AGE mig-heavy-7f46f99c6b-gbvtq 1/1 Running 0 2m2s mig-light-6c95b9648b-497t9 1/1 Running 0 114s mig-light-6c95b9648b-bfvzd 1/1 Running 0 114s ``` If the pods start running, run the following command, which shows the MIG UUID and the MIG profile the pods are using. ```bash for pod in $(kubectl get pods \ -l 'app in (mig-heavy,mig-light)' \ -o name); do echo "--- $pod ---" kubectl exec "$pod" -- nvidia-smi -L done ``` You can see each deployment's pods are using different profiles. ```bash --- pod/mig-heavy-7f46f99c6b-gbvtq --- GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-808c2abc-6e32-babd-e395-27e1ce85be2e) MIG 3g.40gb Device 0: (UUID: MIG-2bb86c61-14b3-54a4-b66f-7ce790800659) --- pod/mig-light-6c95b9648b-497t9 --- GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-808c2abc-6e32-babd-e395-27e1ce85be2e) MIG 1g.10gb Device 0: (UUID: MIG-1d5247a0-b4c5-58b2-9ca5-8e887d6a2441) --- pod/mig-light-6c95b9648b-bfvzd --- GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-808c2abc-6e32-babd-e395-27e1ce85be2e) MIG 1g.10gb Device 0: (UUID: MIG-cba658f3-fb61-52a8-8079-0ed3f1470615) ``` ## Clean Up First, delete the two deployments created for the mig-heavy and mig-light example. ```bash kubectl delete deployment mig-heavy kubectl delete deployment mig-light ``` Then, clean up the operator pods using the following command. ```bash helm uninstall gpu-operator -n gpu-operator ``` ## Conclusion In this guide, we have configured GPU sharing strategies like Time-Slicing, MPS, and MIG. And we have tested each strategy with an example deployment to see how they work. We cannot use a single sharing strategy for all types of workloads. Choose based on what your workloads need. For example, if your workload needs hardware-level isolation, use MIG.