---
title: "Batch Size"
description: "1 posts about Batch Size, summarised, each linking to the original."
---

# Batch Size
> 1 posts about Batch Size, summarised, each linking to the original.

## Articles

### [The Design Space of Tri-Modal Masked Diffusion Models](https://yomu.fyi/post/the-design-space-of-tri-modal-masked-diffusion-models.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Oct 14, 2026
- Type: Announcement

Discrete diffusion models have emerged as strong alternatives to autoregressive language models, but prior approaches typically initialize a unimodal model and fine-tune for bimodal generation. This work diverges from that paradigm by introducing the first tri-modal Masked Diffusion Model pretrained from scratch jointly on text, image-text, and audio-text data, and systematically analyzing multimodal scaling laws, modality mixing ratios, noise schedules, and batch-size effects. A key contribution is a stochastic differential equation reparameterization that decouples physical and logical batch sizes, removing the need for batch size tuning reported in recent work. The authors pretrain a 3B-parameter model on 6.4 trillion tokens and provide optimized inference sampling defaults, achieving strong results on text generation, text-to-image, and text-to-speech tasks. This work represents the largest open systematic study of multimodal discrete diffusion models conducted to date.