Loading…
Topic
4 posts about Multilingual. Every summary links to the original.
See the idea
Arabic is not a single target for a language model. This explainer maps its four parts — dialect surface, data pipeline, model, and evaluation — so you can tell where an Arabic feature really succeeds or fails.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Code-switching, the alternation between languages in a single utterance, is common in multilingual speech but underrepresented in training corpora for speech models. To address this, a lightweight method detects code-switching directly from speech without requiring multilingual transcriptions. The approach leverages language-selective dimensions in pretrained speech encoders by identifying hidden-state dimensions selective to each language from monolingual recordings and aggregating frame-level activations to predict code-switching. On the CS-FLEURS dataset covering 14 language pairs, the Whisper-small encoder achieves a mean F1 of 0.798, outperforming a CNN baseline (0.776) and a HuBERT linear baseline (0.751). These results show that pretrained speech encoders inherently encode the information necessary for code-switching detection, offering a scalable tool for building more representative multilingual corpora.
In data-constrained language model pre-training, two remedies exist: aggressive hyperparameter tuning such as high weight decay, and mixing in data from a high-resource auxiliary language to expand the training distribution. This study systematically compares these levers across four model scales from 150M to 1.43B parameters, using Arabic as the low-resource target and English as the auxiliary, over approximately 1000 pre-training runs. Mixing yields larger improvements than hyperparameter tuning on both validation loss and downstream task accuracy, and the gap grows with model size. Mixing boosts performance by an amount equivalent to 2–3× the unique target data on validation loss and 2–13× on downstream task accuracy, with the gain scaling steeply with model size. The authors recommend mixing in a high-resource language, prioritizing the mixing ratio over hyperparameter tuning, and transferring hyperparameters from a small proxy model via μP, noting that target-language validation loss systematically underestimates mixing’s value.
A year after arguing that AI sovereignty means choice, Cloudflare adds two European open models—EuroLLM and Apertus—to Workers AI, reflecting a year where access to frontier models became constrained and attacks using AI increased. EuroLLM supports 35 languages including all 24 EU official languages, many underserved by existing open models, and was trained on the MareNostrum 5 supercomputer. Apertus, Switzerland’s first fully open multilingual model, was trained on 15 trillion tokens across 1,500+ languages on the Alps supercomputer (10,000+ GH200 GPUs) and outperforms leading models on rare and regional languages. Cloudflare also launches hands-on workshops for government cyber agencies and critical infrastructure operators to build model-agnostic AI security defenses, starting in Singapore in October. The company open-sourced its own security harness that coordinates multiple AI models for vulnerability hunting, ensuring defenses remain operational even if access to one model is lost.
The open-source TTS model landscape has exploded, with over 8,000 models on Hugging Face Hub, yet evaluation methods remain unstandardized and cannot scale. The Open TTS Leaderboard addresses this by using objective metrics such as word error rate (WER), speaker similarity (SIM), and speed (RTFx and TTFA) to evaluate models in hours instead of weeks. It supports multilingual evaluation and voice cloning, providing separate tabs for listening to outputs and comparing streaming performance. Unlike arena-based leaderboards that significantly underrepresent open models due to high hosting costs, this leaderboard prioritizes open-source models. The project encourages community participation through feedback and voting; evaluation scripts will be open-sourced soon, and future updates will incorporate new datasets, languages, and metrics.
The best engineering reads, once a week.
