NVIDIA DGX Spark now supports eight-node clusters, scaling unified memory to 1 TB for local frontier-class model serving like GLM 5.3 (753B) at ~80 tokens/s, using NVIDIA Sync and Cluster Assistant for easy clustering. A Spark-optimized collective backend, RoCEnante, reduces tensor-parallel inference latency on switch-connected topologies, delivering a 4.43x decode speedup at eight nodes—a 45% scaling efficiency gain over baseline, as benchmarked with Qwen3.8-27B model. Users can pick a switchless ring for simpler four-node setups or a switch for eight nodes and multi-group workloads, with both topologies benchmarked. NeMo Switchyard routes easy requests to MiniCPM5 2B and hard ones to Qwen, boosting aggregate throughput by 1.5x to 3x depending on traffic split while matching Qwen-only quality on a tool-use benchmark. Pre-training nanochat (896.5M parameters) achieves near-linear scaling with 97.4% efficiency across eight nodes, reaching 68,079 tokens per second and enabling local training before scaling out, with code available on GitHub.
Context
AI developers need to run and orchestrate multiple frontier-class models requiring hundreds of gigabytes of memory, previously not feasible on single workstations. NVIDIA DGX Spark, originally a single-node system, now scales to eight nodes with official support at GTC Berlin, providing 1 TB unified memory.
Approach / What changed
NVIDIA Sync with Cluster Assistant enables easy clustering and device management. A Spark-optimized collective backend, RoCEnante, reduces communication latency for tensor-parallel inference on switch-connected topologies via all-to-all exchange. Users can choose a switchless ring topology for simpler four-node setups or a switched configuration for eight nodes and multi-group workloads. NeMo Switchyard provides multi-model orchestration with a lightweight classifier routing requests to appropriate model pools.
Takeaways
1
Eight-node DGX Spark clusters provide 1 TB unified memory, enabling local serving of frontier-class models like GLM 5.3 (753B) at interactive speeds (~80 tokens/s).
2
The RoCEnante backend improves tensor-parallel decode scaling by up to 45% on switch-connected clusters, but is limited to direct any-to-all topologies and not a replacement for NCCL in larger or high-concurrency setups.
3
Multi-model orchestration with NeMo Switchyard can boost aggregate throughput by 1.5x to 3x by routing easy requests to a smaller model (MiniCPM5 2B) while preserving quality on complex tasks.
FAQ
How much unified memory does an eight-node DGX Spark cluster provide?
An eight-node DGX Spark cluster provides 1 TB of unified memory (128 GB per node * 8). This allows loading frontier-class models such as the full GLM 5.3 (753B parameters) that would not start on a four-node system.
What is the benefit of the RoCEnante backend over NCCL?
RoCEnante implements an all-to-all exchange that reduces communication steps for small-message, low-concurrency scenarios on switch-connected clusters, delivering up to a 4.43x decode speedup across eight nodes. However, it exchanges more data and is not suitable for high concurrency or larger clusters where NCCL's bandwidth efficiency wins.
Can I use a switchless ring topology for eight nodes?
No, a switchless ring topology is only effective for up to four nodes. For eight nodes, split cluster allocations, or the RoCEnante backend, a switch is required to provide direct any-to-any connectivity between every node.
A tradeoff map for synthetic audiobooks: natural-language style control buys character differentiation, but spends captioned data, tokens, and compute. See which lever to pull first and when plain narration is good enough.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.