Loading post…
AppleProblem & solution
Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
Summary
Multimodal search agents typically rely on freeform reasoning before acting, but small models often produce lengthy reasoning that offers little guidance and increases inference cost. The authors propose Selection-Based Structured Reasoning (SSR), which reframes reasoning as selecting from pre-specified natural-language candidates rather than open-ended generation. SSR enables parallel scoring by using teacher-forced prefilling and a shared context KV cache to compute token likelihoods across candidates simultaneously. Evaluated on seven multimodal search benchmarks with 2B and 4B models, SSR maintains task performance under various reinforcement learning objectives and supervised fine-tuning. The framework achieves search success rates competitive with leading agents of the same scale while reducing per-turn reasoning latency by over 90% and total per-question inference latency by 28–54%.
Context
Multimodal agents commonly generate freeform reasoning before each action. For small models, limited model capacity can result in lengthy reasoning that provides little useful guidance for action generation while incurring substantial inference cost.
Approach / What changed
SSR reformulates reasoning as selection instead of open-ended generation. It represents recurring high-level reasoning as pre-specified, reusable natural-language candidates. At each turn, the model selects from these candidates based on their likelihoods given the current context, without an auxiliary task head. Pre-specified reasoning traces enable parallel scoring, where teacher-forced prefilling computes token likelihoods concurrently within and across candidates using a shared context KV cache.
Takeaways
SSR avoids lengthy, unguided freeform reasoning from small models by converting reasoning into a selection task from predefined natural-language candidates.
The framework achieves high efficiency through parallel scoring of reasoning candidates using teacher-forced prefilling and a shared context KV cache.
On 2B and 4B models across seven multimodal search benchmarks, SSR matches the success rate of similarly scaled agents while cutting per-turn reasoning latency by over 90% and total inference latency by 28–54%.
FAQ
How does SSR improve reasoning efficiency for small models?
SSR replaces open-ended reasoning generation with selection from pre-specified, reusable natural-language candidates. By using teacher-forced prefilling and a shared context KV cache, it computes token likelihoods for multiple candidates in parallel, dramatically reducing per-turn reasoning latency and total inference cost.
What benchmarks and model sizes were used to evaluate SSR?
SSR was evaluated on seven multimodal search benchmarks using models with 2B and 4B parameters. The evaluation covered multiple reinforcement learning objectives and supervised fine-tuning, consistently showing competitive success rates with leading agents of the same scale.
What are the reported latency reductions from using SSR?
SSR reduces per-turn reasoning latency by over 90% and total per-question model inference latency by 28–54%, while maintaining task performance comparable to similarly scaled search agents.
Source and freshness
- Source
- machinelearning.apple.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.