Transformers v5: Simple model definitions powering the AI ecosystem

Summary

Transformers v5.0.0rc-0 introduces major architectural updates focused on simplicity, training, inference, and ecosystem interoperability across modern AI workflows. The release adopts a modular modeling approach and centralizes attention implementations into a unified AttentionInterface abstraction to reduce contribution and code review overhead. Support for Flax and TensorFlow is officially sunset in favor of focusing on PyTorch as the primary backend, while tokenization is standardized around the tokenizers library. For execution workloads, v5 adds native continuous batching, paged attention mechanisms, and a dedicated transformers serve OpenAI-compatible serving system. Finally, weight loading is refactored to make low-precision quantization a first-class citizen alongside broad interoperability with formats such as GGUF, MLX, and TorchAO.

Context

Five years after the release of Transformers v4, the library expanded from 40 model architectures to over 400 and exceeded 1.2 billion total installs. To maintain relevance as a core model definition toolkit across diverse tools, the project needed to streamline code maintenance, reduce contribution complexity, and modernize support for large-scale training, inference, and quantization across the AI ecosystem.

Approach / What changed

Transformers v5 adopts a modular design, introduces an AttentionInterface abstraction, and sunsets Flax and TensorFlow support to focus on PyTorch. The release standardizes tokenization around the tokenizers backend, eliminating the fast/slow tokenizer distinction. It also expands pre-training compatibility, integrates continuous batching, paged attention, and a new transformers serve API, while making quantization a first-class feature alongside cross-ecosystem formats like GGUF.

Takeaways

• Transformers v5 drops backend support for Flax and TensorFlow to focus exclusively on PyTorch as its primary modeling backend.
• The AttentionInterface centralizes attention methods like FlashAttention, FlexAttention, and SDPA, keeping only eager evaluation in core modeling files.
• The library removes the fast and slow tokenizer distinction, standardizing primarily on the tokenizers backend with torchvision for fast image processors.

Related reading

• Hugging Face · Sep 11, 2025

  Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers

  To support OpenAI's GPT-OSS series of models, the transformers library introduced several performance upgrades that apply across supported architectures. The release integrates zero-build custom kernels downloadable directly from the Hub, reducing external dependency bloat and compilation friction for operations like Liger RMSNorm and MegaBlocks MoE. Native support for MXFP4 quantization groups vector elements into 32-value blocks with shared scales, allowing GPT-OSS 20B to fit in roughly 16 GB of VRAM and GPT-OSS 120B in roughly 80 GB. In addition, transformers incorporates Flash Attention 3 with attention sinks, continuous batching via the generate_batch API for experimentation, and automatic memory pre-allocation to speed up model loading on GPUs.

  Aritra Roy Gosthipaty, Sergio Paniego, Vaibhav Srivastav, Pedro Cuenca, Arthur Zucker, Nathan Habib, Cyril Vallez

• huggingface.co · Sep 24, 2021

  Summer at Hugging Face

  Hugging Face released several major updates across its ecosystem, expanding its public Hub repository to over 16,000 models. Platform additions include Spaces for deploying Gradio and Streamlit demo applications, automatic TensorBoard instances, and evaluation metric tracking integrated with Papers With Code leaderboards. The Transformers library gained JAX/Flax support across more than 5,000 models, improved TensorFlow implementations, and introduced the transformers.onnx export module for model conversion. In research, the BigScience project completed large-scale training of a 13-billion-parameter English decoder model on Jean Zay, while the DeDLOC method enabled training the sahajBERT Bengali model without HPC infrastructure. Accepted conference papers detailed the Datasets library, prompt data-point equivalence, prompt-based fine-tuning heuristics, and block pruning techniques that reduced BERT size by 74 percent.