# Making automatic speech recognition work on large files with Wav2Vec2 in 🤗 Transformers

huggingface.co · Nicolas Patry · Feb 1, 2022

**Type:** Problem & solution

## Summary

Transformer-based automatic speech recognition models like Wav2Vec2 crash on long audio inputs because the O(n²) attention mechanism rapidly exhausts GPU memory. Simple audio chunking avoids out-of-memory errors but causes poor transcription accuracy at chunk boundaries due to a lack of surrounding context. To resolve this, Hugging Face Transformers leverages the Connectionist Temporal Classification architecture to process overlapping audio chunks with configurable strides. The pipeline discards low-quality logits at chunk edges and chains the remaining central predictions together to reconstruct a seamless transcript. This striding technique operates out of the box with language model-augmented pipelines and adapts directly to low-latency live audio streaming.

## Context

Running Wav2Vec2 directly on long audio files causes out-of-memory crashes because transformer attention complexity scales quadratically with sequence length. While naive chunking prevents crashes, lack of context around cut boundaries degrades recognition quality, and silence- or voice-detection workarounds are unreliable.

## Approach / What changed

Hugging Face Transformers leverages the Connectionist Temporal Classification (CTC) architecture to process overlapping chunks using configurable stride parameters (chunk\_length\_s and stride\_length\_s). The pipeline generates logits for overlapping segments, discards the boundary logits where accuracy is low, and chains the central logits together before decoding.

## Takeaways

- Transformer attention complexity scales quadratically with sequence length, causing unchunked Wav2Vec2 speech recognition inference to crash with out-of-memory errors on long audio files.
- Connectionist Temporal Classification (CTC) maps each audio frame to a letter prediction logit, allowing pipelines to drop edge logits from overlapping chunks and concatenate the high-quality central outputs.
- The chunking and striding method applies directly to language model-augmented Wav2Vec2 checkpoints and enables live streaming inference by processing overlapping incoming audio frames.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Python](https://yomu.fyi/topic/python), [Streaming](https://yomu.fyi/topic/streaming)

[Read original post](https://huggingface.co/blog/asr-chunking)