# Transformers v5: Simple model definitions powering the AI ecosystem

[Hugging Face](https://yomu.fyi/company/hugging-face) · Lysandre, Arthur Zucker, Cyril Vallez, Vaibhav Srivastav · Dec 1, 2025

**Type:** Announcement

## Summary

Transformers v5.0.0rc-0 introduces major architectural updates focused on simplicity, training, inference, and ecosystem interoperability across modern AI workflows. The release adopts a modular modeling approach and centralizes attention implementations into a unified AttentionInterface abstraction to reduce contribution and code review overhead. Support for Flax and TensorFlow is officially sunset in favor of focusing on PyTorch as the primary backend, while tokenization is standardized around the tokenizers library. For execution workloads, v5 adds native continuous batching, paged attention mechanisms, and a dedicated transformers serve OpenAI-compatible serving system. Finally, weight loading is refactored to make low-precision quantization a first-class citizen alongside broad interoperability with formats such as GGUF, MLX, and TorchAO.

## Context

Five years after the release of Transformers v4, the library expanded from 40 model architectures to over 400 and exceeded 1.2 billion total installs. To maintain relevance as a core model definition toolkit across diverse tools, the project needed to streamline code maintenance, reduce contribution complexity, and modernize support for large-scale training, inference, and quantization across the AI ecosystem.

## Approach / What changed

Transformers v5 adopts a modular design, introduces an AttentionInterface abstraction, and sunsets Flax and TensorFlow support to focus on PyTorch. The release standardizes tokenization around the tokenizers backend, eliminating the fast/slow tokenizer distinction. It also expands pre-training compatibility, integrates continuous batching, paged attention, and a new transformers serve API, while making quantization a first-class feature alongside cross-ecosystem formats like GGUF.

## Takeaways

- Transformers v5 drops backend support for Flax and TensorFlow to focus exclusively on PyTorch as its primary modeling backend.
- The AttentionInterface centralizes attention methods like FlashAttention, FlexAttention, and SDPA, keeping only eager evaluation in core modeling files.
- The library removes the fast and slow tokenizer distinction, standardizing primarily on the tokenizers backend with torchvision for fast image processors.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Python](https://yomu.fyi/topic/python)

[Read original post](https://huggingface.co/blog/transformers-v5)