Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

Summary

Visual chain-of-thought methods allow multimodal large language models to reason through intermediate image generation, but synthesizing explicit visual steps creates major inference overhead during proactive video reasoning. To eliminate runtime synthesis, Internalized Visual Thinking introduces a post-training framework that jointly optimizes textual prediction and next-embedding prediction across unlabeled videos. By predicting latent representations of future frames alongside target text answers during training, the model captures motion, object transitions, interactions, and latent intent. At inference time, the model produces answers directly without generating or re-encoding future frames. IVT outperforms text-only post-training across six evaluation settings while matching or exceeding Visual CoT performance and cutting end-to-end latency by more than 5x.

Takeaways

1. 1

   Internalized Visual Thinking jointly optimizes textual prediction and next-embedding prediction during post-training, learning latent representations of future frames from unlabeled video.

2. 2

   IVT generates final answers directly at inference time without synthesizing or re-encoding intermediate future frames in pixel space.

3. 3

   Compared to Visual CoT, IVT delivers comparable or superior performance across evaluations while reducing end-to-end latency by more than 5x.