What Is an AI Factory? Power In, Tokens Out
An AI factory is a fixed power and cooling budget converted into model output. This explainer maps its four parts and the throughput-per-watt trade-offs that connect them.
Loading post…
NVIDIASarah McKenneyBenchmark
How NVIDIA DSX MaxLPS Maximizes AI Factory Throughput and Efficiency
Summary
AI factories often underutilize power capacity due to static provisioning for peak GPU draw. NVIDIA DSX MaxLPS uses policy-governed dynamic power allocation to reclaim stranded capacity. In a joint evaluation with Nscale using Kimi K2.5 workloads on GB300 NVL72 systems, DSX MaxLPS enabled 37% more GPUs (192 vs 140) within the same 264.4 kW budget. Aggregate throughput increased 49.2% to 1.62 million tokens/s, while per-instance throughput remained unchanged and median latency stayed within 5% of baseline. However, P99 time to first token increased 17%, highlighting the need to evaluate tail latency alongside capacity gains.
Context
AI factories are typically provisioned for the unlikely moment when every GPU reaches peak power, creating a protective buffer that leaves valuable infrastructure underused during normal operation. Static per-node reservations prevent unused capacity from being applied to other nodes, so the aggregate facility can remain below its limit while additional GPU capacity stays offline.
Approach / What changed
DSX MaxLPS combines chip, system, thermal, and software technologies with a control loop that maps topology, collects telemetry, applies operator-defined policies, dynamically adjusts GPU power limits, and validates compliance. The evaluation used 35 four-GPU nodes (140 GPUs) as a static baseline and 48 four-GPU nodes (192 GPUs) with DSX MaxLPS, running Kimi K2.5 FP4 inference workloads on NVIDIA GB300 NVL72 systems at Nscale’s Iceland data center.
Takeaways
Static power provisioning for peak GPU draw leaves significant capacity unused; DSX MaxLPS reclaims this by dynamically reallocating power within a fixed budget.
In the evaluation, adding 37% more GPUs increased aggregate throughput 49.2% without reducing per-instance throughput, but P99 time to first token rose 17%.
Operators should validate DSX MaxLPS with a staged process: define the managed boundary, establish a baseline, introduce policies conservatively, add capacity incrementally, and set production limits.
FAQ
How does DSX MaxLPS increase GPU count without exceeding the power budget?
DSX MaxLPS monitors real-time power consumption and dynamically reallocates unused headroom from underutilized resources to additional GPUs, staying within the operator-defined aggregate budget. This enables more GPUs to operate productively beneath the same managed power limit.
What trade-offs were observed in the evaluation?
Median and P75 latency remained within 5% of baseline, but P99 time to first token increased 17% from 15.7 seconds. This shows that stable average performance can mask tail-latency degradation, which operators must evaluate alongside capacity gains.
What validation steps should operators follow for DSX MaxLPS?
Operators should define the managed boundary, run a representative baseline under static provisioning, introduce policies conservatively, add capacity incrementally while comparing performance at each step, and approve only when throughput, latency, and power compliance are met under normal and adverse conditions.
Source and freshness
- Source
- developer.nvidia.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Visual explainers
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.