STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation

Summary

Existing multimodal models struggle with structural fragmentation, often sacrificing visual fidelity via discrete tokenization or breaking structural symmetry by pairing causal text generation with diffusion-based denoising. STARFlow2 addresses these limitations by leveraging autoregressive normalizing flows, which share the causal mask, key-value cache mechanism, and left-to-right structure of large language models. Built upon the Pretzel architecture, the system vertically interleaves a frozen pretrained vision-language model stream with a TARFlow stream through residual skip connections under a single causal mask. A deep-shallow flow design combined with a unified FAE latent space allows both text and continuous visual outputs to enter the key-value cache directly without re-encoding. Benchmark experiments validate strong performance across multimodal understanding and continuous image generation tasks.

Context

Unified multimodal models that understand, reason over, and generate interleaved text-image sequences remain structurally fragmented. Existing methods either sacrifice visual fidelity through discrete tokenization, impose structural asymmetry by combining causal text generation with iterative diffusion-based denoising, or degrade pretrained understanding when adapting vision-language models for generation.

Approach / What changed

STARFlow2 adopts the Pretzel architecture to vertically interleave a frozen pretrained vision-language model stream with a TARFlow stream via residual skip connections under a single causal mask. It couples a deep-shallow flow design with a unified FAE latent space to enable cache-friendly interleaved generation where text and visual outputs enter the KV-cache directly without re-encoding.

Takeaways

• Autoregressive normalizing flows share the same causal mask, KV-cache mechanism, and left-to-right structure as LLMs, making them suitable for continuous, single-pass multimodal generation.
• The Pretzel architecture vertically interleaves a frozen pretrained VLM stream with a TARFlow stream via residual skip connections, preserving understanding while generating images.
• Combining a deep-shallow flow design with a unified FAE latent space allows text and visual outputs to enter the KV-cache directly without requiring re-encoding.

Related reading

• Apple · Aug 24, 2026

  Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

  Visual chain-of-thought methods allow multimodal large language models to reason through intermediate image generation, but synthesizing explicit visual steps creates major inference overhead during proactive video reasoning. To eliminate runtime synthesis, Internalized Visual Thinking introduces a post-training framework that jointly optimizes textual prediction and next-embedding prediction across unlabeled videos. By predicting latent representations of future frames alongside target text answers during training, the model captures motion, object transitions, interactions, and latent intent. At inference time, the model produces answers directly without generating or re-encoding future frames. IVT outperforms text-only post-training across six evaluation settings while matching or exceeding Visual CoT performance and cutting end-to-end latency by more than 5x.

• Netflix · Jul 17, 2026

  In-House LLM Serving at Netflix

  Netflix established an in-house serving platform to run large language model inference directly inside existing production environments alongside traditional machine learning models. Built upon NVIDIA Triton Inference Server and vLLM, the unified architecture handles member-scale routing, candidate generation, feature fetching, and model execution over gRPC and OpenAI-compatible HTTP endpoints. The team shifted their primary engine from TensorRT-LLM to vLLM to support custom architectures, simpler debugging, and non-trivial constraint logic. Addressing production obstacles required patching Triton's frontend for guided decoding, pinning dependent library versions, rewriting logits processors in C++ for vLLM V1, and handling state machine resets during engine preemptions. The resulting platform unifies deployment flows while preserving operational stability across evolving model schemas.