---
title: "Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot"
description: "NVIDIA Snapshot is a new open-source component of the NVIDIA DSX AI Factory Platform that checkpoints and restores initialized GPU workloads on Kubernetes, skipping repeated cold-start phases such as weight loading, compilation, and CUDA graph capture. Benchmarks on NVIDIA B200 GPUs with VAST NFS storage show restore times of 3.6 to 42.7 seconds across vLLM, SGLang, and TensorRT LLM, achieving 8x to 36x speedups over cold starts. For large mixture-of-experts models with 1 TB checkpoints across eight GPUs, Snapshot restored replicas in under 43 seconds compared with cold-start times of 6 to 19 minutes. Checkpoint creation took 15 seconds to 5.5 minutes, but checkpoint time plus restore time was shorter than a single cold start in every configuration. Platform teams integrate Snapshot via Kubernetes custom resources and the annotation nvidia.com/restore-from."
---

# Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Ron Kahn · Nov 10, 2026

**Type:** Benchmark

## Summary

NVIDIA Snapshot is a new open-source component of the NVIDIA DSX AI Factory Platform that checkpoints and restores initialized GPU workloads on Kubernetes, skipping repeated cold-start phases such as weight loading, compilation, and CUDA graph capture. Benchmarks on NVIDIA B200 GPUs with VAST NFS storage show restore times of 3.6 to 42.7 seconds across vLLM, SGLang, and TensorRT LLM, achieving 8x to 36x speedups over cold starts. For large mixture-of-experts models with 1 TB checkpoints across eight GPUs, Snapshot restored replicas in under 43 seconds compared with cold-start times of 6 to 19 minutes. Checkpoint creation took 15 seconds to 5.5 minutes, but checkpoint time plus restore time was shorter than a single cold start in every configuration. Platform teams integrate Snapshot via Kubernetes custom resources and the annotation nvidia.com/restore-from.

## Context

When inference demand spikes or a replica fails, available GPUs do not translate immediately into serving capacity because initializing an LLM workload (a cold start) can take minutes, delaying scaling and recovery. NVIDIA Snapshot is a new open-source component of the NVIDIA DSX AI Factory Platform that makes initialized GPU workloads reusable across inference replicas, speeding startup and recovery.

## Approach / What changed

NVIDIA Snapshot is a checkpoint and restore system for GPU workloads on Kubernetes. It captures a fully initialized workload, including CPU and GPU memory, as a portable artifact and restores that state on compatible infrastructure. Platform teams integrate it via Kubernetes custom resources: a PodSnapshot names a running Pod to checkpoint, a SnapshotJob checkpoints a Pod as soon as it is ready, and the annotation nvidia.com/restore-from on a new Pod triggers a restore. The workload performs a small handshake to pause before checkpoint and resume after restore.

## Takeaways

- NVIDIA Snapshot restores initialized GPU workloads in 3.6 to 42.7 seconds, 8x to 36x faster than cold starts, across vLLM, SGLang, and TensorRT LLM on NVIDIA B200 GPUs.
- Checkpoint time plus restore time was shorter than a single cold start in every tested configuration, making the one-time checkpoint cost worthwhile for the first restored replica.
- For large MoE models with 1 TB checkpoints across eight GPUs, Snapshot restored replicas in under 43 seconds, while cold starts took 6 to 19 minutes.

## FAQ

### How does Snapshot skip cold-start phases?

Snapshot captures a fully initialized workload, including CPU and GPU memory, as a portable artifact. Restore skips weight loading, compilation, autotuning, and CUDA graph capture, bringing inference capacity online in seconds instead of minutes.

### Does checkpointing add overhead that offsets gains?

Checkpoint time plus restore time was shorter than a single cold start in every tested configuration. Checkpoint creation took 15 seconds to 5.5 minutes, depending on the model and configuration.

### Which inference engines are supported?

Workloads must meet Snapshot's checkpoint/restore and runtime compatibility requirements. The repository has ready-made examples for vLLM, SGLang, and NVIDIA TensorRT LLM.

**Tags:** [Checkpoint/Restore](https://yomu.fyi/topic/checkpoint-restore), [GPU](https://yomu.fyi/topic/gpu), [Inference](https://yomu.fyi/topic/inference), [Kubernetes](https://yomu.fyi/topic/kubernetes), [NVIDIA](https://yomu.fyi/topic/nvidia)

- Source: [NVIDIA](https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot)
- Source URL: https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot
- Ingested by Yomu: 2026-11-10T18:01:11.118Z

[Read original post](https://developer.nvidia.com/blog/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot)
