---
title: "Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning"
description: "The open-source TTS model landscape has exploded, with over 8,000 models on Hugging Face Hub, yet evaluation methods remain unstandardized and cannot scale. The Open TTS Leaderboard addresses this by using objective metrics such as word error rate (WER), speaker similarity (SIM), and speed (RTFx and TTFA) to evaluate models in hours instead of weeks. It supports multilingual evaluation and voice cloning, providing separate tabs for listening to outputs and comparing streaming performance. Unlike arena-based leaderboards that significantly underrepresent open models due to high hosting costs, this leaderboard prioritizes open-source models. The project encourages community participation through feedback and voting; evaluation scripts will be open-sourced soon, and future updates will incorporate new datasets, languages, and metrics."
---

# Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning

[Hugging Face](https://yomu.fyi/company/hugging-face) · Eric Bezzam, Steven Zheng, Eustache Le Bihan, mrfakename · Sep 30, 2026

**Type:** Announcement

## Summary

The open-source TTS model landscape has exploded, with over 8,000 models on Hugging Face Hub, yet evaluation methods remain unstandardized and cannot scale. The Open TTS Leaderboard addresses this by using objective metrics such as word error rate (WER), speaker similarity (SIM), and speed (RTFx and TTFA) to evaluate models in hours instead of weeks. It supports multilingual evaluation and voice cloning, providing separate tabs for listening to outputs and comparing streaming performance. Unlike arena-based leaderboards that significantly underrepresent open models due to high hosting costs, this leaderboard prioritizes open-source models. The project encourages community participation through feedback and voting; evaluation scripts will be open-sourced soon, and future updates will incorporate new datasets, languages, and metrics.

## Context

TTS model releases have outpaced evaluation methods. Existing arena-based leaderboards rely on human preference votes, which cannot scale and underrepresent open-source models due to hosting costs.

## Approach / What changed

Open TTS Leaderboard uses objective metrics (WER, CER, RTFx, TTFA, SIM) to evaluate TTS models on intelligibility, speed, and speaker similarity. It supports multilingual and voice cloning evaluation, and provides a ‘Listen’ tab for comparing outputs. Evaluation can be done in hours rather than weeks.

## Takeaways

- Open TTS Leaderboard evaluates models using WER/CER for intelligibility, RTFx and TTFA for speed, and SIM for speaker similarity.
- English performance does not translate to other languages; the leaderboard supports multilingual evaluation with CER for character-based languages like Chinese and Japanese.
- The leaderboard includes a ‘Listen’ tab for comparing audio outputs and a ‘Streaming’ tab for comparing time-to-first-audio (TTFA) performance.

## FAQ

### Why use objective metrics instead of human preference?

Human preference scores cannot scale to keep up with TTS releases. Objective metrics like WER and RTFx allow evaluation in hours rather than weeks and can serve as a proxy for intelligibility and speed.

### How does the leaderboard handle multilingual evaluation?

It uses WER for languages with word-level errors and CER for character-based languages like Chinese, Japanese, and Korean. English performance does not guarantee performance in other languages.

### Can the community contribute to the leaderboard?

Yes, the leaderboard encourages community feedback and will open-source evaluation scripts. Users can also vote on outputs in the 'Listen' tab after logging in with a Hugging Face account to help weed out spam.

**Tags:** [Benchmark](https://yomu.fyi/topic/benchmark), [Multilingual](https://yomu.fyi/topic/multilingual), [Text-to-Speech](https://yomu.fyi/topic/text-to-speech), [Voice Cloning](https://yomu.fyi/topic/voice-cloning)

- Source: [Hugging Face](https://huggingface.co/blog/open-tts-leaderboard)
- Source URL: https://huggingface.co/blog/open-tts-leaderboard
- Ingested by Yomu: 2026-09-30T15:00:09.225Z

[Read original post](https://huggingface.co/blog/open-tts-leaderboard)