---
title: "How to Run Isolated Tenant Kubernetes Clusters on Shared GPU Infrastructure"
description: "Organizations sharing a single Kubernetes cluster across multiple teams frequently encounter conflicting custom resource definitions, overlapping role-based access control, and difficulties budgeting GPU capacity. To address these operational conflicts without provisioning separate hardware, administrators can deploy vCluster alongside KAI Scheduler on a shared host cluster. vCluster supplies each team with a dedicated, virtualized control plane featuring its own API server and permissions, while KAI Scheduler dynamically time-slices GPU resources through hierarchical queues. In a demonstration using an NVIDIA L40S GPU on MicroK8s, three separate tenant clusters successfully scheduled workloads with fractional allocations of 0.33 GPU each. The combination achieves tenant autonomy and visibility isolation while consolidating AI workloads on shared physical infrastructure."
---

# How to Run Isolated Tenant Kubernetes Clusters on Shared GPU Infrastructure

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Tanya Lenz · Aug 3, 2026

**Type:** Tutorial

## Summary

Organizations sharing a single Kubernetes cluster across multiple teams frequently encounter conflicting custom resource definitions, overlapping role-based access control, and difficulties budgeting GPU capacity. To address these operational conflicts without provisioning separate hardware, administrators can deploy vCluster alongside KAI Scheduler on a shared host cluster. vCluster supplies each team with a dedicated, virtualized control plane featuring its own API server and permissions, while KAI Scheduler dynamically time-slices GPU resources through hierarchical queues. In a demonstration using an NVIDIA L40S GPU on MicroK8s, three separate tenant clusters successfully scheduled workloads with fractional allocations of 0.33 GPU each. The combination achieves tenant autonomy and visibility isolation while consolidating AI workloads on shared physical infrastructure.

## Context

Managing multiple teams on a single shared Kubernetes cluster introduces coordination friction, such as conflicting CRD versions, overlapping RBAC rules, and an inability to partition GPU capacity into team-level budgets. However, spinning up dedicated physical clusters per team creates unnecessary infrastructure overhead.

## Approach / What changed

Deploy virtual tenant clusters using vCluster on top of a shared host cluster while utilizing KAI Scheduler and the NVIDIA GPU Operator to partition GPU compute. vCluster provides each team with an isolated control plane, and KAI Scheduler uses hierarchical Queue CRDs to assign fractional GPU quotas across tenant workloads running on the same hardware.

## Takeaways

- KAI Scheduler handles pod scheduling and fractional GPU allocation via queue hierarchies, but it does not enforce hardware-level GPU memory isolation under software time-slicing.
- Pods target KAI Scheduler by specifying kai-scheduler as their schedulerName, alongside queue labels and gpu-fraction annotations, allowing kube-scheduler to handle standard workloads.
- vCluster isolates API servers, custom resource definitions, namespaces, and access controls for each team while sharing underlying worker nodes and GPU resources.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Kubernetes](https://yomu.fyi/topic/kubernetes), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source)

- Source: [NVIDIA](https://developer.nvidia.com/blog/how-to-run-isolated-tenant-kubernetes-clusters-on-shared-gpu-infrastructure)
- Source URL: https://developer.nvidia.com/blog/how-to-run-isolated-tenant-kubernetes-clusters-on-shared-gpu-infrastructure
- Ingested by Yomu: 2026-08-28T04:00:49.410Z

[Read original post](https://developer.nvidia.com/blog/how-to-run-isolated-tenant-kubernetes-clusters-on-shared-gpu-infrastructure)