Easily Build and Share ROCm Kernels with Hugging Face

Summary

Compiling custom deep learning GPU kernels across architectures often leads to complex build flag management, compiler errors, and ABI issues. Hugging Face's kernels library and kernel-builder streamline this workflow by supporting backends such as ROCm, CUDA, Metal, and XPU. Using the RadeonFlow FP8 block-wise GEMM kernel optimized for the AMD Instinct MI300X GPU as an example, developers organize files into specific headers and HIP implementations. The build is configured via a build.toml manifest and flake.nix to ensure reproducible compilation across supported PyTorch and ROCm versions. Artifacts can then be pushed to the Hugging Face Hub, allowing users to load and execute custom operators directly via get_kernel without traditional installation steps.

Context

Compiling custom GPU kernels for targeted hardware architectures, managing build flags, and integrating with PyTorch extensions frequently results in CMake or Nix configuration issues, compiler errors, and ABI mismatches.

Approach / What changed

Hugging Face's kernels library and kernel-builder utilize build.toml manifests and flake.nix reproducibility files to compile ROCm HIP code, package PyTorch C++ bindings, and publish artifacts to the Hugging Face Hub for direct loading.

Takeaways

• The RadeonFlow GEMM kernel uses the FP8 e4m3fnuz format with per-block FP32 scaling factors to maintain numerical accuracy on AMD Instinct MI300X (gfx942) GPUs.
• Kernel source files must distinguish between .h files for headers/declarations and .hip files for HIP implementation code that compiles separately.
• Published kernels do not require traditional installation and can be loaded directly from the Hugging Face Hub using the get_kernel function.

Related reading

• Hugging Face · Sep 11, 2025

  Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers

  To support OpenAI's GPT-OSS series of models, the transformers library introduced several performance upgrades that apply across supported architectures. The release integrates zero-build custom kernels downloadable directly from the Hub, reducing external dependency bloat and compilation friction for operations like Liger RMSNorm and MegaBlocks MoE. Native support for MXFP4 quantization groups vector elements into 32-value blocks with shared scales, allowing GPT-OSS 20B to fit in roughly 16 GB of VRAM and GPT-OSS 120B in roughly 80 GB. In addition, transformers incorporates Flash Attention 3 with attention sinks, continuous batching via the generate_batch API for experimentation, and automatic memory pre-allocation to speed up model loading on GPUs.

  Aritra Roy Gosthipaty, Sergio Paniego, Vaibhav Srivastav, Pedro Cuenca, Arthur Zucker, Nathan Habib, Cyril Vallez

• huggingface.co · Mar 16, 2022

  Image search with 🤗 datasets

  Hugging Face datasets expanded its capabilities by introducing an Image feature type, enabling image processing and integration with vector indexing tools. The library was applied to a sample of historical book embellishments extracted via OCR from the British Library. Images were loaded using the ImageFolder loader, enriched with filename metadata, and pushed to the Hugging Face Hub. A FAISS index and CLIP embeddings were used to retrieve images matching natural language queries, such as categories, specific objects, and boolean operators. While the retrieval system demonstrated reasonable semantic search results across various prompts, full public deployment was avoided due to CLIP model card restrictions and potential bias in the historical dataset.