# Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions

[Apple](https://yomu.fyi/company/apple) · Aug 20, 2026

**Type:** Problem & solution

## Summary

Cross-lingual knowledge transfer is critical for training multilingual language models when target-language data is scarce and downstream tasks demand reasoning or world knowledge. Existing transfer approaches often depend on scarce resources such as large parallel corpora, translation systems, auxiliary models, or extra training phases. To address this limitation, the authors introduce LINK, a data-level intervention method that operates directly during pretraining via bilingual vocabularies. LINK swaps randomly selected words in a portion of the high-resource English corpus with word-level translations according to a specified replacement ratio. Across evaluations spanning eight languages and five model sizes, the technique delivered notable performance gains on downstream target-language tasks while achieving up to a 2x speedup in training to reach equivalent performance.

## Context

Building high-performing multilingual language models for languages with scarce training data requires transferring knowledge from high-resource languages for tasks like scientific reasoning, commonsense inference, and world knowledge. Existing methods for cross-lingual knowledge transfer depend on large parallel corpora, full translation systems, auxiliary models, or additional training stages that are unavailable for many low-resource languages.

## Approach / What changed

The authors propose LINK, a data-level intervention method applied during model pretraining. LINK uses low-cost bilingual vocabularies to perform lexical substitutions within the high-resource English pretraining corpus. Based on a specified replacement ratio, randomly selected English words in a portion of the corpus are swapped with their word-level translations without requiring auxiliary models or extra training stages.

## Takeaways

- Existing cross-lingual transfer approaches rely heavily on translation systems, parallel data, or auxiliary models that are largely unavailable for data-scarce target languages.
- LINK performs random word-level substitutions in the high-resource training corpus using bilingual vocabularies that can be acquired at near-zero cost for almost any language.
- In evaluations across eight languages and five model sizes, LINK delivered notable improvements on target-language downstream tasks and achieved up to a 2x training speedup to reach equivalent performance.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning)

[Read original post](https://machinelearning.apple.com/research/multilingual-knowledge-transfer-lexical-interventions)