Loading…
Topic
RDMA
3 posts about RDMA. Every summary links to the original.
- 3 posts
- 3 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
How DOCA GPUNetIO Unifies GPU-Initiated Networking Across the NVIDIA Software Stack
NVIDIA DOCA GPUNetIO is a GPU-centric networking SDK that enables CUDA kernels to drive Ethernet, RDMA, Verbs, and DMA operations with the CPU removed from the application critical path. It unifies previously fragmented GDA-KI-style RDMA implementations from libraries such as NCCL, NVSHMEM, UCX, and the Aerial 5G SDK into a single shared foundation. The framework provides both a full DOCA SDK version and a lighter open-source Verbs-focused implementation; the open-source version can optionally dlopen the SDK at runtime. On the data path, CUDA kernels directly post Work Queue Entries, ring the NIC doorbell, and poll for completions without CPU intervention. Benchmarks using NVSHMEM show that the GPU-initiated path avoids the CPU proxy bottleneck, achieving higher bandwidth for small message sizes compared to CPU-mediated communication.
Modal Clusters are generally available
Modal announced general availability of Modal Clusters, a serverless multi-node compute primitive for AI workloads that provides up to 6.4 Tbps InfiniBand RDMA per node with PyTorch and NCCL support. A new gang scheduler places N nodes simultaneously by observing the fleet, assigning all pending clusters, and provisioning capacity when needed. RDMA support is fully automated behind a single rdma=True flag, with Modal contributing RDMA support to the gVisor container runtime and upstreaming the changes. Customers like Decagon, 1x, and Runway use Clusters for trillion-parameter fine-tuning, pre-training home robot world models, and multi-node video inference respectively. Clusters are available to all workspaces today through the @modal.clustered decorator, with per-second billing and no reservation required.
MetaRoCE: A New RDMA Transport Built for AI-Scale Ethernet
Meta has designed MetaRoCE, a clean-sheet RDMA transport for AI workloads running on commodity Ethernet, targeting clusters that can reach million-GPU scale. Unlike standard RoCE, it treats packet loss and out-of-order delivery as normal: packets are sprayed across NIC-managed paths, written directly to their destinations, and retransmitted selectively without PFC or reorder buffers. It combines per-path telemetry and congestion windows with ECN-based sender control and receiver-driven fair-share hints, while a single connection supports multiple ordered streams and paths. On a 64-node AMD GPU cluster using RCCL collectives, Meta reports higher throughput and lower flow-completion times than RoCEv2, about 86% throughput at 1% packet loss, linear scaling across four- and eight-plane topologies, and autonomous recovery from plane failures. Meta is contributing the specification, reference implementation, and compliance suite to OCP, with a broader release planned for October at the 2026 OCP Global Summit.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.