Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR

Summary

Code-switching automatic speech recognition faces major performance challenges due to the scarcity of code-switching training data. To mitigate this data limitation, an iterative pseudo-labeling framework leverages a large unlabeled audio corpus to generate semi-supervised speech datasets. The system structure executes across three sequential phases: pseudo-label generation, two-stage bilingual model pre-training with supervised code-switching fine-tuning, and progressive iterative improvements. Successive refinement cycles further enhance the neural model's recognition accuracy when processing complex code-switching scenarios across alternating languages. When evaluated on the SEAME benchmark, this training pipeline achieved Mix Error Rate reductions of 6.35 percent on the devman subset and 8.29 percent on the devsge subset.

Context

Code-switching, alternating languages within the same utterance, presents major challenges for automatic speech recognition systems due to limited code-switching training data.

Approach / What changed

An iterative pseudo-labeling training framework for code-switching automatic speech recognition structured in three phases: pseudo-label generation from a large unlabeled corpus, two-stage bilingual model pre-training and supervised code-switching fine-tuning, and iterative progressive refinements.

Takeaways

• Iterative pseudo-labeling generates semi-supervised training data from large unlabeled corpora to address data scarcity in code-switching ASR.
• The bilingual model architecture employs a two-stage training strategy comprising pre-training on semi-supervised data followed by fine-tuning on supervised code-switching data.
• The progressive refinement framework reduced Mix Error Rate by 6.35% on SEAME's devman subset and 8.29% on the devsge subset.

Related reading

• Apple · Aug 20, 2026

  Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions

  Cross-lingual knowledge transfer is critical for training multilingual language models when target-language data is scarce and downstream tasks demand reasoning or world knowledge. Existing transfer approaches often depend on scarce resources such as large parallel corpora, translation systems, auxiliary models, or extra training phases. To address this limitation, the authors introduce LINK, a data-level intervention method that operates directly during pretraining via bilingual vocabularies. LINK swaps randomly selected words in a portion of the high-resource English corpus with word-level translations according to a specified replacement ratio. Across evaluations spanning eight languages and five model sizes, the technique delivered notable performance gains on downstream target-language tasks while achieving up to a 2x speedup in training to reach equivalent performance.

• Apple · Aug 19, 2026

  Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts