STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation

Summary

Existing multimodal models struggle with structural fragmentation, often sacrificing visual fidelity via discrete tokenization or breaking structural symmetry by pairing causal text generation with diffusion-based denoising. STARFlow2 addresses these limitations by leveraging autoregressive normalizing flows, which share the causal mask, key-value cache mechanism, and left-to-right structure of large language models. Built upon the Pretzel architecture, the system vertically interleaves a frozen pretrained vision-language model stream with a TARFlow stream through residual skip connections under a single causal mask. A deep-shallow flow design combined with a unified FAE latent space allows both text and continuous visual outputs to enter the key-value cache directly without re-encoding. Benchmark experiments validate strong performance across multimodal understanding and continuous image generation tasks.

Takeaways

1. 1

   Autoregressive normalizing flows share the same causal mask, KV-cache mechanism, and left-to-right structure as LLMs, making them suitable for continuous, single-pass multimodal generation.

2. 2

   The Pretzel architecture vertically interleaves a frozen pretrained VLM stream with a TARFlow stream via residual skip connections, preserving understanding while generating images.

3. 3

   Combining a deep-shallow flow design with a unified FAE latent space allows text and visual outputs to enter the KV-cache directly without requiring re-encoding.