# 20x Faster TRL Fine-tuning with RapidFire AI

[Hugging Face](https://yomu.fyi/company/hugging-face) · Kamran Bigdely, Arun Kumar, Quentin Gallouédec · Nov 21, 2025

**Type:** Announcement

## Summary

Hugging Face TRL now officially integrates with RapidFire AI to accelerate LLM fine-tuning and post-training experimentation through concurrent configuration evaluation. Instead of executing training configurations sequentially, the platform shards datasets into discrete chunks and cycles multiple model configurations across GPUs at chunk boundaries. The runtime handles automatic checkpointing via a shared-memory-based adapter and model spilling mechanism to maintain consistent training state. Experimenters can monitor live metrics on an MLflow-based dashboard and use Interactive Control Ops to stop underperforming runs or clone and warm-start promising configurations mid-flight. Benchmark results on NVIDIA A100 GPUs demonstrate experimentation speedups ranging from 15x to 20x over sequential runs when training TinyLlama-1.1B and Llama-3.2-1B models.

## Context

Fine-tuning and post-training LLMs sequentially across multiple configurations wastes time and GPU cycles, leaving teams with insufficient budget or bandwidth to compare hyperparameters despite the potential for significant metric improvements.

## Approach / What changed

RapidFire AI integrates into Hugging Face TRL using drop-in config replacements (RFSFTConfig, RFDPOConfig, RFGRPOConfig) and an adaptive chunk-based scheduler. The scheduler shards datasets into chunks, cycles configurations across GPUs at chunk boundaries via shared-memory model and adapter spilling, and enables real-time Interactive Control Ops from a dashboard to stop or clone runs mid-flight.

## Takeaways

- RapidFire AI provides drop-in wrappers including RFSFTConfig, RFDPOConfig, and RFGRPOConfig to replace standard TRL configurations with minimal code changes.
- The scheduler shards datasets into chunks and alternates configurations at chunk boundaries, enabling early comparative metrics and reaching 95%+ GPU utilization.
- Mid-flight Interactive Control Ops allow users to stop, resume, delete, or clone and warm-start configurations directly from the dashboard without restarting jobs.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance), [Python](https://yomu.fyi/topic/python)

[Read original post](https://huggingface.co/blog/rapidfireai)