# Open ASR Leaderboard: Trends and Insights with New Multilingual & Long-Form Tracks

[Hugging Face](https://yomu.fyi/company/hugging-face) · Eric Bezzam, Steven Zheng, Eustache Le Bihan, Vaibhav Srivastav · Nov 21, 2025

**Type:** Benchmark

## Summary

The Open ASR Leaderboard expanded its evaluation framework by introducing dedicated multilingual and long-form transcription tracks across eleven datasets. Benchmarking more than sixty open- and closed-source models from eighteen organizations revealed clear operational trade-offs between accuracy and inference speed. Systems combining Conformer encoders with large language model decoders achieved the lowest word error rates in short-form English speech recognition. In contrast, connectionist temporal classification and token-and-duration transducer decoders yielded tenfold to hundredfold improvements in throughput with only slight accuracy penalties. While closed-source commercial systems maintained performance leads in long-form audio transcription, specialized single-language models consistently outperformed generalized systems supporting hundreds of languages.

## Context

Most speech recognition benchmarks focus on short-form English transcription under 30 seconds, neglecting critical variables such as multilingual performance and inference throughput for long-form audio like meetings and podcasts.

## Approach / What changed

The Open ASR Leaderboard incorporated multilingual and long-form transcription tracks, comparing more than 60 open- and closed-source models from 18 organizations across 11 datasets using word error rate and inverse real-time factor metrics.

## Takeaways

- Combining Conformer or Fast Conformer encoders with LLM decoders achieves top English transcription accuracy, as demonstrated by Canary-Qwen-2.5B, Granite-Speech-3.3-8B, and Phi-4-Multimodal-Instruct.
- CTC and TDT decoders deliver 10x to 100x faster throughput than LLM decoders, making them suitable for real-time or batch transcription despite slightly higher error rates.
- Closed-source systems maintain a lead over open-source alternatives in long-form transcription, while fine-tuned or single-language models consistently outperform massively multilingual architectures in accuracy.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://huggingface.co/blog/open-asr-leaderboard)