---
title: "Scaling Agentic RL: High-Throughput Agentic Training with Tunix"
description: "Tunix is presented as a post-training library for scaling agentic reinforcement learning, where multi-turn tool use and environment latency can leave TPUs idle. Its design combines an asyncio-based, high-concurrency trajectory collector with asynchronous vLLM-TPU and SGLang-Jax integration, then streams variable-length trajectories through a producer-consumer queue rather than waiting for a complete synchronous batch. Composable agent and environment abstractions expose APIs for custom tools, benchmarks, and external systems while preserving multi-turn token handling and lifecycle management. Lightweight RL-specific metrics correlate rollout, training, weight synchronization, model-call, and environment stages with TPU timelines, complementing targeted XProf traces. The stated outcome is a more observable pipeline intended to reduce stalls, keep accelerators utilized, and support agentic RL natively in the JAX/TPU ecosystem."
---

# Scaling Agentic RL: High-Throughput Agentic Training with Tunix

[Google](https://yomu.fyi/company/google) · Haoyu Gao, Lance Wang, Shadi Noghabi, Tianshu Bao, Weiren Yu · Jul 21, 2026

**Type:** Explainer

## Summary

Tunix is presented as a post-training library for scaling agentic reinforcement learning, where multi-turn tool use and environment latency can leave TPUs idle. Its design combines an asyncio-based, high-concurrency trajectory collector with asynchronous vLLM-TPU and SGLang-Jax integration, then streams variable-length trajectories through a producer-consumer queue rather than waiting for a complete synchronous batch. Composable agent and environment abstractions expose APIs for custom tools, benchmarks, and external systems while preserving multi-turn token handling and lifecycle management. Lightweight RL-specific metrics correlate rollout, training, weight synchronization, model-call, and environment stages with TPU timelines, complementing targeted XProf traces. The stated outcome is a more observable pipeline intended to reduce stalls, keep accelerators utilized, and support agentic RL natively in the JAX/TPU ecosystem.

## Context

Agentic RL shifts alignment from single-turn interactions to multi-turn reasoning, tool use, and environment interaction. These workflows can leave TPUs waiting on code execution, database queries, web searches, environment steps, or slow trajectories, while synchronous batching and training create additional pipeline stalls. Standard profilers provide detailed traces but are too expensive for continuous monitoring.

## Approach / What changed

Tunix uses asynchronous trajectory collection with Python asyncio, async vLLM-TPU and SGLang-Jax integrations, and a producer-consumer pipeline that streams completed trajectories into the trainer. It separates agent and environment abstractions through composable APIs, supports custom external environments, and adds lightweight RL-specific instrumentation that correlates macro-stage metrics with TPU timelines.

## Takeaways

- The asynchronous collector overlaps model inference, tool execution, and reward computation, reducing execution bubbles and the straggler effect caused by slow trajectories.
- Tunix separates agent behavior from environment lifecycle management, allowing environments such as benchmarks, games, or external systems to be integrated without changing the training workflow.
- Continuous RL-specific metrics expose rollout, training, weight synchronization, model-call, and environment timing; XProf can then be used for targeted operator-level investigation.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents), [JAX](https://yomu.fyi/topic/jax), [Reinforcement Learning](https://yomu.fyi/topic/reinforcement-learning), [vLLM](https://yomu.fyi/topic/vllm)

- Source: [Google](https://developers.googleblog.com/scaling-agentic-rl-high-throughput-agentic-training-with-tunix)
- Source URL: https://developers.googleblog.com/scaling-agentic-rl-high-throughput-agentic-training-with-tunix
- Ingested by Yomu: 2026-09-09T14:28:16.806Z

[Read original post](https://developers.googleblog.com/scaling-agentic-rl-high-throughput-agentic-training-with-tunix)