---
title: "What Is an AI Factory? Four Stages From Data to Served Answer"
description: "An AI factory turns raw data and GPU-hours into served model output. This diagram breaks it into four stages — feed, shape, serve, restore — plus the control plane that owns them."
---

# What Is an AI Factory? Four Stages From Data to Served Answer

**Type:** [What is it?](https://yomu.fyi/visual-explainers/category/what-is-it.md)

![Diagram of an AI factory: a heading and definition above a left-to-right line of four labeled stages — feed, shape, serve, restore — joined by arrows, with a control plane rail spanning underneath and one amber callout about idle GPU time.](https://yomufyi.s3.us-east-1.amazonaws.com/visual-explainers/af20e705-73eb-49c6-a85a-a2745ee5989d/image-309bf424-2d48-4a31-9b69-9ccf5eca61b2.webp)

> An AI factory turns raw data and GPU-hours into served model output. This diagram breaks it into four stages — feed, shape, serve, restore — plus the control plane that owns them.

## Explanation

An AI factory is the production system that converts data and GPU-hours into served model responses, and its real metric is how little of that expensive GPU time sits idle. Four stages do the work: data streams to every GPU, workloads are shaped to the hardware with explicit execution control, replicas serve inference, and initialized workloads are checkpointed so restores take seconds instead of cold starts. A control plane provisions clusters and node images so the team keeps ownership of software, operations, and service. Engineers should care because the bottlenecks rarely sit in the model — they sit in data delivery, cold starts, and idle sessions.

## FAQ

### Where does an AI factory usually bottleneck first, data or compute?

Often data. Databricks reported that on a 300 GB feature table across eight H100 GPUs, streaming Unity Catalog tables directly to the GPUs cut training time roughly 10x compared with reading an exported Parquet copy, because data delivery rather than the model was the constraint.

### What does the control plane actually give the operating team?

NVIDIA's NVKE supplies management-plane APIs and NVKR builds reproducible node images from version-pinned sources, so AI factories retain control of their software, operations, service, and customer experience. Teams can adopt the two projects together or independently.

**Tags:** [AI Factory](https://yomu.fyi/topic/ai-factory.md), [Checkpoint/Restore](https://yomu.fyi/topic/checkpoint-restore.md), [Cost Optimization](https://yomu.fyi/topic/cost-optimization.md), [GPU](https://yomu.fyi/topic/gpu.md), [Inference](https://yomu.fyi/topic/inference.md), [Infrastructure](https://yomu.fyi/topic/infrastructure.md)

## Related reading

### [How We Built TLX: A Hardware-Native GPU Compiler for Production ML at Scale](https://yomu.fyi/post/how-we-built-tlx-a-hardware-native-gpu-compiler-for-production-ml-at-s.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Hongtao Yu, Peng Chen, Daohang Shi, Nick Riasanovsky, Lei Wang, Partha Kanuparthy
- Published: Nov 10, 2026
- Type: Announcement

Triton Low-Level Extensions (TLX) is an extension to Triton that helps engineers optimize GPU performance while preserving Triton’s productive programming model. TLX introduces explicit warp-group and cluster-level orchestration, letting developers adopt new hardware capabilities without sacrificing velocity. Modern GPU architectures increasingly rely on execution-oriented features such as warp specialization, asynchronous memory movement, tensor memory, and cluster-level execution to optimize their performance. TLX has demonstrated performance improvements in RecSys training, RecSys inference, and large-scale LLM pretraining. Across NVIDIA and AMD GPUs, TLX combines state-of-the-art kernels with explicit execution orchestration and model-aware fusion to deliver end-to-end improvements beyond what standalone libraries can achieve.


### [Agent Substrate: Advancing Kubernetes for Agents at Scale](https://yomu.fyi/post/agent-substrate-advancing-kubernetes-for-agents-at-scale.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Ekin Karabulut
- Published: Nov 10, 2026
- Type: Announcement

AI agents spend most of their time idle, a pattern Kubernetes was not designed for, leading to wasted compute or cold-start penalties. Agent Substrate, now a CNCF Sandbox project, is an open-source runtime that multiplexes many agent sessions over a small pool of worker Pods. Idle sessions are checkpointed with memory and filesystem state and restored onto any free worker, avoiding API-server round trips. The project targets coding assistants, RL post-training with agentic environments, and GPU sandboxes for research. NVIDIA is contributing upstream to integrate Agent Substrate with KAI Scheduler and NVIDIA Dynamo for coordinated CPU-side tool execution and GPU-side inference.


### [Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot](https://yomu.fyi/post/start-and-restore-gpu-workloads-in-seconds-with-nvidia-snapshot.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Ron Kahn
- Published: Nov 10, 2026
- Type: Benchmark

NVIDIA Snapshot is a new open-source component of the NVIDIA DSX AI Factory Platform that checkpoints and restores initialized GPU workloads on Kubernetes, skipping repeated cold-start phases such as weight loading, compilation, and CUDA graph capture. Benchmarks on NVIDIA B200 GPUs with VAST NFS storage show restore times of 3.6 to 42.7 seconds across vLLM, SGLang, and TensorRT LLM, achieving 8x to 36x speedups over cold starts. For large mixture-of-experts models with 1 TB checkpoints across eight GPUs, Snapshot restored replicas in under 43 seconds compared with cold-start times of 6 to 19 minutes. Checkpoint creation took 15 seconds to 5.5 minutes, but checkpoint time plus restore time was shorter than a single cold start in every configuration. Platform teams integrate Snapshot via Kubernetes custom resources and the annotation nvidia.com/restore-from.


### [Manage Kubernetes Clusters for GPU Infrastructure with NVIDIA Kubernetes Engine](https://yomu.fyi/post/manage-kubernetes-clusters-for-gpu-infrastructure-with-nvidia-kubernet.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Nathan Taber
- Published: Nov 10, 2026
- Type: Announcement

NVIDIA announced two open source projects, NVIDIA Kubernetes Engine (NVKE) and NVIDIA Kubernetes Recipes (NVKR), for building and operating managed Kubernetes services on GPU infrastructure. NVKE provides the management-plane software with APIs to provision and manage clusters, while NVKR builds reproducible node images, container images, and packages from version-pinned sources. Both are components of the NVIDIA DSX AI Factory Platform and are available on GitHub under the Apache 2.0 license in public preview. The walkthrough covers building a node image with NVKR, deploying the NVKE management plane using nvkectl, and provisioning a customer Kubernetes cluster through the NVKE API. AI factories retain control of their software, operations, service, and customer experience, and teams can adopt the projects together or independently.


### [Accelerate GPU training on Unity Catalog tables with UCDataset](https://yomu.fyi/post/accelerate-gpu-training-on-unity-catalog-tables-with-ucdataset.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: John Spencer, Milan Stefanovic, Bruce Fontaine, Tejas Sundaresan
- Published: Nov 9, 2026
- Type: Announcement

Databricks released UCDataset in Beta on AI Runtime, a native PyTorch data loader that streams Unity Catalog tables directly to every GPU in a training job without exports or custom loader code. The bottleneck in GPU training is often data delivery, not the model, and common workarounds like exporting to Parquet or using toPandas() are slow or do not scale. UCDataset reads a table’s files directly from cloud storage, splits matching rows evenly across all GPUs and workers, and applies filters and deletion vectors as data is read. On a simulated production workload with a 300 GB feature table across 8 NVIDIA H100 GPUs, UCDataset cut training time by 10x compared to reading an exported Parquet copy. The package is available on PyPI and reads Delta tables registered in Unity Catalog, including tables with deletion vectors.
