How to Partition AMD GPUs for Shared AI Compute
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
Loading post…
Red HatSudhakar Molli, Bohdan Dobrelia, Miguel Carpio, Petr Kubica, Venkataramana VenugopalBenchmark
GPU virtualization at scale: AMD MI300X SR-IOV on Red Hat OpenStack Services on OpenShift
Summary
Enterprise AI environments are moving from dedicated GPU servers to shared accelerator infrastructure, requiring higher GPU utilization with strong isolation. AMD Instinct MI300X GPUs support hardware-assisted SR-IOV in CPX mode, each exposing eight virtual functions with separate PCI and VRAM allocation. When integrated with Red Hat OpenStack Services on OpenShift, the control plane runs on OpenShift while RHEL compute nodes use EDPM, with Nova scheduling VFs via VFIO PCI passthrough. Validation on an 8-GPU system created 64 VFs, demonstrated 64 VMs with one VF each, and tested up to 32 VFs per VM. BabelStream benchmarks showed approximately 96-99% of bare-metal memory bandwidth, while the testing also identified limitations in GPU peer-to-peer DMA and runtime VF lifecycle.
Context
Enterprise AI environments are increasingly moving from dedicated GPU servers toward shared accelerator infrastructure that can support multiple users, teams, and workloads on the same hardware. This creates a need to increase GPU utilization while maintaining strong workload isolation and predictable performance.
Approach / What changed
Using AMD GIM driver 8.6.0 in CPX mode to partition each MI300X GPU into 8 SR-IOV virtual functions, combined with Red Hat OpenStack Services on OpenShift where the control plane runs on OpenShift and GPU-equipped RHEL EDPM compute nodes are managed via External Data Plane Management. OpenStack Nova inventories and schedules the VFs as PCI resources, and VFIO attaches them to tenant VMs via PCI passthrough.
Takeaways
BabelStream benchmarks on MI300X SR-IOV VFs delivered approximately 96-99% of bare-metal GPU memory bandwidth for primary operations.
8 MI300X GPUs in CPX mode expose 64 VFs, supporting a maximum-density configuration of 64 VMs each with 1 VF, and configurations with up to 32 VFs assigned to a single VM.
GPU P2P DMA between VFs is blocked by IOMMU isolation, runtime VF unbinding can cause host hangs, and each CPX VF exposes about 24 GB of VRAM.
FAQ
How many VFs can each MI300X GPU expose?
In CPX mode, each MI300X GPU exposes 8 SR-IOV virtual functions. An 8-GPU node therefore provides up to 64 schedulable GPU VFs.
What is the GPU memory bandwidth of a VF compared to bare metal?
BabelStream testing showed approximately 96-99% of bare-metal GPU memory bandwidth for primary operations.
What are the main limitations?
GPU P2P DMA between VFs is blocked by IOMMU isolation, runtime VF unbinding can cause host hangs, and each CPX VF exposes about 24 GB of VRAM. Large PCI configurations up to 32 VFs per VM were tested; 64 VFs per VM is under investigation.
Source and freshness
- Source
- developers.redhat.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Visual explainers
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.