PRX Part 3 — Training a Text-to-Image Model in 24h!

Summary

PRX Part 3 reports a 24-hour speedrun for training a text-to-image diffusion model, combining techniques that performed well in earlier experiments under a budget of about $1,500 on 32 H200 GPUs. The recipe uses x-prediction in pixel space with 32-pixel patches and a 256-dimensional initial bottleneck, followed by LPIPS and DINOv2 perceptual losses, TREAD token routing, REPA alignment with DINOv3, and Muon for matrix parameters while Adam handles the rest. Training ran for 100,000 steps at 512px with batch size 1,024, then 20,000 steps at 1024px with batch size 512 and no REPA, across three synthetic datasets. The resulting model is described as usable, with strong prompt following and consistent aesthetics, though texture glitches, anatomy issues, and difficulty with hard prompts remain. The post attributes these limitations mainly to undertraining and limited data diversity and presents the recipe as a foundation for further scaling.

Takeaways

1. 1

   Pixel-space training uses x-prediction, 32-pixel patches, and a 256-dimensional initial token bottleneck; sequence length is 256 at 512px and 1,024 at 1024px.

2. 2

   TREAD routes 50% of tokens from the second transformer block to the penultimate block, while REPA alignment is applied at block 8 only to non-routed tokens.

3. 3

   After one day, the model has strong prompt following and consistent aesthetics, but still shows texture glitches, occasional strange anatomy, and instability on difficult prompts.