# DeepMath: A lightweight math reasoning Agent with smolagents

[Hugging Face](https://yomu.fyi/company/hugging-face) · Daniel Fleischer, Moshe Berchansky, Moshe Wasserblat · Dec 4, 2025

**Type:** Problem & solution

## Summary

Mathematical reasoning in large language models often suffers from lengthy chain-of-thought traces and frequent arithmetic mistakes. To address these issues, DeepMath pairs a Qwen3-4B Thinking base model with a sandboxed Python execution environment built using the smolagents library and vLLM backend. The system offloads deterministic calculations by emitting concise Python snippets, executing them safely with restricted module imports and no network access, and folding returned values back into the context. Training utilizes Group Relative Policy Optimization on the Tool-Integrated Reasoning subset of OpenMathReasoning with temperature scheduling and length constraints. Across benchmarks including MATH500, AIME, HMMT, and HLE, DeepMath reduces output token lengths by up to 66% while increasing overall problem-solving accuracy.

## Context

Mathematical problem-solving in large language models typically generates lengthy chain-of-thought traces that remain prone to arithmetic and numerical errors. Existing approaches often do not focus on reducing trace verbosity or training models to favor concise, computation-driven traces executed within a constrained, auditable environment.

## Approach / What changed

DeepMath pairs Qwen3-4B Thinking with a sandboxed Python executor using the smolagents library and vLLM inference engine. The model emits short Python snippets that run in a restricted sandbox without file I/O or network access under strict timeouts and module allow-lists, inserting execution results directly into the reasoning trace. DeepMath is fine-tuned using Group Relative Policy Optimization (GRPO) via a modified TRL framework on the Tool-Integrated Reasoning subset of OpenMathReasoning. GRPO training balances an accuracy reward (+1) and a code-use reward (+1, weighted 10:1 vs. accuracy), restricts completion candidates to 5,000 tokens, applies linear temperature scheduling from 1.2 down to 0.7, and leverages four in-context examples.

## Takeaways

- DeepMath integrates a sandboxed Python executor with Qwen3-4B Thinking via smolagents and vLLM, offloading deterministic calculations and reducing output trace lengths by up to 66%.
- GRPO fine-tuning balances a +1 accuracy reward with a +1 code-snippet reward weighted 10:1 vs accuracy, alongside linear temperature scheduling from 1.2 to 0.7 and a 5,000-token candidate cap.
- Benchmarks on MATH500, AIME, HMMT, and HLE show that combining both agentic execution and GRPO training achieves higher accuracy than using either agentic inference or GRPO alone.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Python](https://yomu.fyi/topic/python)

[Read original post](https://huggingface.co/blog/intel-deepmath)