---
title: "Agent Substrate: Advancing Kubernetes for Agents at Scale"
description: "AI agents spend most of their time idle, a pattern Kubernetes was not designed for, leading to wasted compute or cold-start penalties. Agent Substrate, now a CNCF Sandbox project, is an open-source runtime that multiplexes many agent sessions over a small pool of worker Pods. Idle sessions are checkpointed with memory and filesystem state and restored onto any free worker, avoiding API-server round trips. The project targets coding assistants, RL post-training with agentic environments, and GPU sandboxes for research. NVIDIA is contributing upstream to integrate Agent Substrate with KAI Scheduler and NVIDIA Dynamo for coordinated CPU-side tool execution and GPU-side inference."
---

# Agent Substrate: Advancing Kubernetes for Agents at Scale

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Ekin Karabulut · Nov 10, 2026

**Type:** Announcement

## Summary

AI agents spend most of their time idle, a pattern Kubernetes was not designed for, leading to wasted compute or cold-start penalties. Agent Substrate, now a CNCF Sandbox project, is an open-source runtime that multiplexes many agent sessions over a small pool of worker Pods. Idle sessions are checkpointed with memory and filesystem state and restored onto any free worker, avoiding API-server round trips. The project targets coding assistants, RL post-training with agentic environments, and GPU sandboxes for research. NVIDIA is contributing upstream to integrate Agent Substrate with KAI Scheduler and NVIDIA Dynamo for coordinated CPU-side tool execution and GPU-side inference.

## Context

AI agents spend most of their time idle, waiting on prompts, tool results, or model responses. Kubernetes was not built for this pattern: running one Pod per session holds memory for idle sessions, and deleting Pods between turns incurs multi-second cold starts and loses state. Platform teams face a choice between keeping warm Pods (wasting compute) or creating Pods on demand (paying cold start penalties on every step).

## Approach / What changed

Agent Substrate multiplexes many agent sessions over a small pool of worker Pods. Idle sessions are checkpointed (memory and filesystem included) and restored onto whichever worker is free when the next request arrives, avoiding cold starts and reducing idle compute.

## Takeaways

- Agent Substrate multiplexes many agent sessions over a small pool of worker Pods, checkpointing idle sessions and restoring them onto any free worker to avoid cold starts.
- The project targets three workloads: coding assistants, RL post-training with agentic environments, and short-lived GPU sandboxes for research.
- Agent Substrate integrates with KAI Scheduler for workload placement and NVIDIA Dynamo for coordinating inference-state signals between agent execution and model serving.

## FAQ

### What happens when all workers are busy and a new request arrives?

When every worker is busy, the next request is parked in the Agent Substrate router and retried until a worker frees up, with a bounded wait and a cap on how many requests can wait at once. A follow-up ensures requests to already-running actors never get stuck in that line.

### How does Agent Substrate handle GPU state across suspend and resume?

Agent Substrate proposes a vendor-neutral design built on Kubernetes Dynamic Resource Allocation (DRA). Device vendors describe how to pause their devices safely and which device properties a snapshot depends on. The system records those properties with each snapshot and restores the sandbox on a matching device, or falls back to a clean cold start if none is available.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents), [CNCF](https://yomu.fyi/topic/cncf), [GPU](https://yomu.fyi/topic/gpu), [Kubernetes](https://yomu.fyi/topic/kubernetes), [NVIDIA](https://yomu.fyi/topic/nvidia)

- Source: [NVIDIA](https://developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale)
- Source URL: https://developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale
- Ingested by Yomu: 2026-11-10T18:01:12.036Z

[Read original post](https://developer.nvidia.com/blog/agent-substrate-advancing-kubernetes-for-agents-at-scale)
