Easily Build and Share ROCm Kernels with Hugging Face

Summary

Compiling custom deep learning GPU kernels across architectures often leads to complex build flag management, compiler errors, and ABI issues. Hugging Face's kernels library and kernel-builder streamline this workflow by supporting backends such as ROCm, CUDA, Metal, and XPU. Using the RadeonFlow FP8 block-wise GEMM kernel optimized for the AMD Instinct MI300X GPU as an example, developers organize files into specific headers and HIP implementations. The build is configured via a build.toml manifest and flake.nix to ensure reproducible compilation across supported PyTorch and ROCm versions. Artifacts can then be pushed to the Hugging Face Hub, allowing users to load and execute custom operators directly via get_kernel without traditional installation steps.

Takeaways

1. 1

   The RadeonFlow GEMM kernel uses the FP8 e4m3fnuz format with per-block FP32 scaling factors to maintain numerical accuracy on AMD Instinct MI300X (gfx942) GPUs.

2. 2

   Kernel source files must distinguish between .h files for headers/declarations and .hip files for HIP implementation code that compiles separately.

3. 3

   Published kernels do not require traditional installation and can be loaded directly from the Hugging Face Hub using the get_kernel function.