Progressive Refinement: An Iterative Pseudo-Labeling Approach for Mandarin-English Code-Switching ASR

Summary

Code-switching automatic speech recognition faces major performance challenges due to the scarcity of code-switching training data. To mitigate this data limitation, an iterative pseudo-labeling framework leverages a large unlabeled audio corpus to generate semi-supervised speech datasets. The system structure executes across three sequential phases: pseudo-label generation, two-stage bilingual model pre-training with supervised code-switching fine-tuning, and progressive iterative improvements. Successive refinement cycles further enhance the neural model's recognition accuracy when processing complex code-switching scenarios across alternating languages. When evaluated on the SEAME benchmark, this training pipeline achieved Mix Error Rate reductions of 6.35 percent on the devman subset and 8.29 percent on the devsge subset.

Takeaways

1. 1

   Iterative pseudo-labeling generates semi-supervised training data from large unlabeled corpora to address data scarcity in code-switching ASR.

2. 2

   The bilingual model architecture employs a two-stage training strategy comprising pre-training on semi-supervised data followed by fine-tuning on supervised code-switching data.

3. 3

   The progressive refinement framework reduced Mix Error Rate by 6.35% on SEAME's devman subset and 8.29% on the devsge subset.