---
title: "RDMA"
description: "4 posts about RDMA, summarised, each linking to the original."
---

# RDMA
> 4 posts about RDMA, summarised, each linking to the original.

## Articles

### [Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio](https://yomu.fyi/post/local-and-distributed-large-llm-inference-with-mlx-and-rdma-on-mac-stu.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Oct 15, 2026
- Type: Benchmark

MLX enables running large language models locally on Apple silicon by exploiting the unified memory architecture of Mac Studio. The latest MLX release supports distributed inference across multiple Macs using RDMA over Thunderbolt, with pipeline or tensor parallelization schemes, and uses the jaccl communication collective. Benchmarks compare M3 Ultra and M5 Ultra across three models: Qwen 3.8 (27B) and Mistral Medium 3.5 (128B) fit on a single 512GB Mac, while the newly released Muse Spark Open (816B) requires parallelization across up to four machines at its 4-bit NVFP4 quantization. The M5 Ultra’s GPU Neural Accelerator delivers up to 4.6 times faster time-to-first-token than the M3 Ultra, while its 1.2 TB/s memory bandwidth improves generation throughput by 1.6 times for dense models. On four M5 Ultra Mac Studios with tensor parallelism, Muse Spark Open achieves over 36 tokens per second.


### [How DOCA GPUNetIO Unifies GPU-Initiated Networking Across the NVIDIA Software Stack](https://yomu.fyi/post/how-doca-gpunetio-unifies-gpu-initiated-networking-across-the-nvidia-s.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Oct 6, 2026
- Type: Explainer

NVIDIA DOCA GPUNetIO is a GPU-centric networking SDK that enables CUDA kernels to drive Ethernet, RDMA, Verbs, and DMA operations with the CPU removed from the application critical path. It unifies previously fragmented GDA-KI-style RDMA implementations from libraries such as NCCL, NVSHMEM, UCX, and the Aerial 5G SDK into a single shared foundation. The framework provides both a full DOCA SDK version and a lighter open-source Verbs-focused implementation; the open-source version can optionally dlopen the SDK at runtime. On the data path, CUDA kernels directly post Work Queue Entries, ring the NIC doorbell, and poll for completions without CPU intervention. Benchmarks using NVSHMEM show that the GPU-initiated path avoids the CPU proxy bottleneck, achieving higher bandwidth for small message sizes compared to CPU-mediated communication.


### [Modal Clusters are generally available](https://yomu.fyi/post/modal-clusters-are-generally-available.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Author: Peyton Walters
- Published: Oct 1, 2026
- Type: Announcement

Modal announced general availability of Modal Clusters, a serverless multi-node compute primitive for AI workloads that provides up to 6.4 Tbps InfiniBand RDMA per node with PyTorch and NCCL support. A new gang scheduler places N nodes simultaneously by observing the fleet, assigning all pending clusters, and provisioning capacity when needed. RDMA support is fully automated behind a single rdma=True flag, with Modal contributing RDMA support to the gVisor container runtime and upstreaming the changes. Customers like Decagon, 1x, and Runway use Clusters for trillion-parameter fine-tuning, pre-training home robot world models, and multi-node video inference respectively. Clusters are available to all workspaces today through the @modal.clustered decorator, with per-second billing and no reservation required.


### [MetaRoCE: A New RDMA Transport Built for AI-Scale Ethernet](https://yomu.fyi/post/metaroce-a-new-rdma-transport-built-for-ai-scale-ethernet.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Arvind Srinivasan, Neil Spring, Omar Baldonado, Rajiv Krishnamurthy
- Published: Aug 24, 2026
- Type: Announcement

Meta has designed MetaRoCE, a clean-sheet RDMA transport for AI workloads running on commodity Ethernet, targeting clusters that can reach million-GPU scale. Unlike standard RoCE, it treats packet loss and out-of-order delivery as normal: packets are sprayed across NIC-managed paths, written directly to their destinations, and retransmitted selectively without PFC or reorder buffers. It combines per-path telemetry and congestion windows with ECN-based sender control and receiver-driven fair-share hints, while a single connection supports multiple ordered streams and paths. On a 64-node AMD GPU cluster using RCCL collectives, Meta reports higher throughput and lower flow-completion times than RoCEv2, about 86% throughput at 1% packet loss, linear scaling across four- and eight-plane topologies, and autonomous recovery from plane failures. Meta is contributing the specification, reference implementation, and compliance suite to OCP, with a broader release planned for October at the 2026 OCP Global Summit.