# Diverse reasoning traces teach LLMs to make better decisions

[Amazon](https://yomu.fyi/company/amazon) · Sheng Jia, Xiao Wang, Shiva Kasiviswanathan · May 26, 2026

**Type:** Explainer

## Summary

Large language models typically rely on supervised fine-tuning with a single human-verified reasoning trace per example, which limits reasoning breadth and causes mode collapse when attempting parallel reasoning. Researchers presented set-supervised fine tuning (SSFT) and global forking policy optimization (GFPO) to teach models diverse problem-solving strategies. SSFT represents reasoning as a set of complete solution paths paired with global forking tokens, utilizing bipartite matching to achieve order-invariant training and specialized tokens. GFPO applies lightweight reinforcement learning to select the optimal forking token upfront, detaching gradients from generated rollouts to ensure stable optimization. On standard reasoning and coding benchmarks, the combined framework increased single-shot accuracy by 5% to 7% while improving pass@k diversity without degrading pass@1 accuracy.

## Context

Large language models are typically trained on a single reasoning trace per example, which causes mode collapse when using naive supervised fine-tuning to elicit diverse parallel reasoning strategies.

## Approach / What changed

Researchers introduced set-supervised fine tuning (SSFT), which uses min-cost bipartite matching to assign multiple reasoning traces to global forking tokens, followed by global forking policy optimization (GFPO) to train token selection using reward advantages with detached rollout gradients.

## Takeaways

- Set-supervised fine tuning uses bipartite matching between traces and global forking tokens to prevent mode collapse during post-training.
- Global forking policy optimization updates the policy over forking tokens using reward signals while detaching gradients from the generated reasoning traces.
- Combining SSFT and GFPO yields 5% to 7% gains in single-shot accuracy on benchmarks including AIME 2024, AIME 2025, and LiveCodeBench-v5.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning)

[Read original post](https://www.amazon.science/blog/diverse-reasoning-traces-teach-llms-to-make-better-decisions)