Hugging FaceHanna Yukhymenko, Oleg Lavrovsky, Y MetzAnnouncement
Apertus 1.5 extends the 8B and 70B open-weight models with image and audio understanding, a thinking mode, and a 262,144-token context window, released under Apache 2.0. The models are a continued pretraining of Apertus 1.0, adding 4 trillion tokens to the 8B model and 2 trillion to the 70B model, for totals of 19T and 17T respectively. Multimodality uses an early-fusion tokenizer with Emu3.5 for images and WavTokenizer for audio, keeping the architecture unchanged and training at text-only speed. Post-training includes SFT, long-context SFT, reinforcement learning with verifiable rewards, DPO, and charter alignment via SDPO, with all data and recipes publicly documented. Evaluation results show competitive performance on reasoning, coding, and multilingual tasks, with the 8B model matching GPT-OSS-20B on translation and outperforming Claude Haiku 4.5, reported transparently via the EvalEval Coalition’s EEE schema.
Apertus 1.5 is a continued pretraining of the open-weight Apertus 1.0 models, adding multimodal capabilities and a thinking mode while maintaining full transparency in training data, recipes, and evaluation.
The models use an early-fusion tokenizer approach, tokenizing images with Emu3.5 and audio with WavTokenizer into discrete tokens in the same sequence as text. Pre-training continues from Apertus 1.0 in four stages: unpaired, paired, cooldown, and long-context. Post-training includes SFT, long-context SFT, RLVR, DPO, and SDPO for charter alignment.
Apertus 1.5 introduces multimodality via an early-fusion tokenizer that tokenizes images and audio into discrete tokens in the same sequence as text, keeping the architecture unchanged and training at text-only speed.
The post-training pipeline includes five stages: SFT, long-context SFT, reinforcement learning with verifiable rewards, DPO, and charter alignment via SDPO, with all training data and recipes publicly documented.
Evaluation results show the 8B model matches GPT-OSS-20B on the Last Translation Benchmark and outperforms Claude Haiku 4.5, while the thinking mode improves performance on reasoning-heavy benchmarks like AIME.
It uses an early-fusion tokenizer approach, tokenizing images with Emu3.5 (512x512 image becomes 1024 tokens) and audio with WavTokenizer (40 tokens per second) into discrete tokens in the same sequence as text, without a separate vision encoder.
The thinking mode can be switched on to reason about the input before answering, which improves performance on reasoning problems. It is trained by mixing thinking on and off during RLVR.
Apertus 1.5 collaborates with the EvalEval Coalition to publish results in the Every Eval Ever (EEE) schema, which records scores, run details, generation settings, and metric interpretation, making evaluations reproducible and comparable.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Compare two ways to adapt a model to a new language or domain: generate a broad synthetic corpus like AI4Bharat's, or fine-tune on one narrow task and evaluate it.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
