---
title: "Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models"
description: "Code-switching, the alternation between languages in a single utterance, is common in multilingual speech but underrepresented in training corpora for speech models. To address this, a lightweight method detects code-switching directly from speech without requiring multilingual transcriptions. The approach leverages language-selective dimensions in pretrained speech encoders by identifying hidden-state dimensions selective to each language from monolingual recordings and aggregating frame-level activations to predict code-switching. On the CS-FLEURS dataset covering 14 language pairs, the Whisper-small encoder achieves a mean F1 of 0.798, outperforming a CNN baseline (0.776) and a HuBERT linear baseline (0.751). These results show that pretrained speech encoders inherently encode the information necessary for code-switching detection, offering a scalable tool for building more representative multilingual corpora."
---

# Code-Switching Detection via Language-Selective Dimensions in Speech Foundation Models

[Apple](https://yomu.fyi/company/apple) · Oct 14, 2026

**Type:** Explainer

## Summary

Code-switching, the alternation between languages in a single utterance, is common in multilingual speech but underrepresented in training corpora for speech models. To address this, a lightweight method detects code-switching directly from speech without requiring multilingual transcriptions. The approach leverages language-selective dimensions in pretrained speech encoders by identifying hidden-state dimensions selective to each language from monolingual recordings and aggregating frame-level activations to predict code-switching. On the CS-FLEURS dataset covering 14 language pairs, the Whisper-small encoder achieves a mean F1 of 0.798, outperforming a CNN baseline (0.776) and a HuBERT linear baseline (0.751). These results show that pretrained speech encoders inherently encode the information necessary for code-switching detection, offering a scalable tool for building more representative multilingual corpora.

## Context

Code-switching utterances are common in multilingual speech but scarce in training data, motivating the need for efficient detection methods to improve corpus diversity.

## Approach / What changed

The method identifies language-selective hidden-state dimensions in pretrained speech encoders (e.g., Whisper-small) using monolingual recordings. It aggregates frame-level activations of these dimensions to predict whether both languages are present in an utterance.

## Takeaways

- Code-switching detection can be performed directly from speech without text transcription by using language-selective dimensions in pretrained encoders.
- On CS-FLEURS across 14 language pairs, Whisper-small achieves mean F1 0.798, outperforming CNN (0.776) and HuBERT linear (0.751) baselines.
- The lightweight method requires only a small code-switching set to tune a decision rule, enabling scalable curation of diverse multilingual training data.

## FAQ

### How does the method detect code-switching without text transcription?

It identifies language-selective hidden-state dimensions in pretrained speech encoders using monolingual recordings, then aggregates frame-level activations to classify whether both languages appear in an utterance.

### What performance does the Whisper-small encoder achieve on CS-FLEURS?

It achieves a mean F1 of 0.798 across 14 language pairs, compared to 0.776 for a CNN baseline and 0.751 for a HuBERT linear baseline.

**Tags:** [Code-Switching](https://yomu.fyi/topic/code-switching), [Multilingual](https://yomu.fyi/topic/multilingual), [Speech Recognition](https://yomu.fyi/topic/speech-recognition)

- Source: [Apple](https://machinelearning.apple.com/research/code-switching-detection)
- Source URL: https://machinelearning.apple.com/research/code-switching-detection
- Ingested by Yomu: 2026-10-14T18:00:13.674Z

[Read original post](https://machinelearning.apple.com/research/code-switching-detection)