NVIDIAElizabeth GoodmanExplainer
AI factories are power-limited systems where GPU workload placement is a key optimization; poor placement fragments topology domains and forces traffic across shared links, reducing throughput and raising job costs. NVIDIA Topograph discovers cluster topology from cloud APIs or on-premises fabric systems, normalizes it into a common model, and publishes it in formats each workload manager expects, such as Kubernetes node labels, Slurm topology configuration, or Slinky ConfigMaps. It uses five components: API Server, Node Observer, Node Data Broker, Provider, and Engine to keep the topology view current as the cluster changes. The post covers deploying Topograph with Helm on Kubernetes, as a system package on Slurm, and as a Helm chart on Slinky, with support for cloud providers including Google Cloud, Lambda, Nebius, Nscale, and OCI. Topograph works alongside Dynamic Resource Allocation and KAI Scheduler to enable topology-aware gang scheduling, improving AI factory efficiency and tokens per watt.
AI factories are power-limited systems where GPU workload placement significantly impacts throughput and cost. Poor placement fragments topology domains and forces traffic across shared links, reducing performance. NVIDIA Topograph addresses this by providing schedulers with a current, provider-reported map of the physical network, enabling topology-aware decisions without manual maintenance.
Topograph is an open source toolkit that discovers cluster topology from cloud APIs or on-premises fabric systems, normalizes it into a canonical model, and publishes it in scheduler-ready formats. It uses providers for discovery and engines for output translation. Deployment options include Helm on Kubernetes, system packages on Slurm, and Helm on Slinky. The system comprises five components: API Server, Node Observer, Node Data Broker, Provider, and Engine, which keep the topology view current as the cluster changes.
Topograph publishes topology as Kubernetes node labels, Slurm topology configuration, or Slinky ConfigMaps, enabling schedulers to make topology-aware decisions without manual maintenance.
Topograph uses five components (API Server, Node Observer, Node Data Broker, Provider, Engine) to keep the topology view current as the cluster changes.
Supported cloud providers include Google Cloud, Lambda, Nebius, Nscale, and OCI, with on-premises support for InfiniBand and Spectrum-X fabrics.
Topograph uses five components: API Server validates and dispatches requests, Node Observer watches cluster changes, Node Data Broker collects per-node attributes, Provider converts cloud or fabric data into a canonical representation, and Engine writes the output in scheduler-ready formats. This pipeline regenerates the topology view on request and upon watched changes.
Topograph outputs Kubernetes node labels, NFD resources, Slinky ConfigMaps, Slurm topology configuration (tree or block), and instance-oriented topology JSON. The format depends on the engine selected: k8s, slurm, slinky, nfd, or instance.
Supported cloud providers include Google Cloud, Lambda, Nebius, Nscale, and Oracle Cloud Infrastructure (OCI). On-premises deployments support InfiniBand in Kubernetes, InfiniBand on bare metal or VMs, and Spectrum-X or NetQ-managed fabric.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
BioNeMo Inference Runtime runs one pipeline per target: parse, tokenize, generate features, infer on a GPU replica, and write a structure. This diagram traces those stages and the throughput trade-off.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
