---
title: "Five things you can do with Sentry’s Agent Tracing"
description: "Sentry’s Agent Tracing captures prompts, tool calls, and outputs in a single trace, enabling debugging of silent failures, slow steps, and context loss in multi-agent workflows. The post details five recipes: exporting production conversations into eval datasets via a CLI command that records the first user message as input and the last agent reply as output. Another recipe automates error triage by having Claude’s scheduled tasks pull stats via the MCP server and file tickets in Linear for broken tools, stuck loops, or costly runs. Additional recipes cover tracking per-call costs with the Vercel AI SDK, building dashboards and alerts for spend monitoring, and sending OpenTelemetry spans to Sentry as an extra destination without replacing existing pipelines."
---

# Five things you can do with Sentry’s Agent Tracing

[Sentry](https://yomu.fyi/company/sentry) · Anton Bjorkman · Oct 28, 2026

**Type:** Tutorial

## Summary

Sentry’s Agent Tracing captures prompts, tool calls, and outputs in a single trace, enabling debugging of silent failures, slow steps, and context loss in multi-agent workflows. The post details five recipes: exporting production conversations into eval datasets via a CLI command that records the first user message as input and the last agent reply as output. Another recipe automates error triage by having Claude’s scheduled tasks pull stats via the MCP server and file tickets in Linear for broken tools, stuck loops, or costly runs. Additional recipes cover tracking per-call costs with the Vercel AI SDK, building dashboards and alerts for spend monitoring, and sending OpenTelemetry spans to Sentry as an extra destination without replacing existing pipelines.

## Context

AI-powered apps are hard to debug without observability because AI produces different results each run. Agent Tracing bridges the gap between prompts, tool calls, and outputs in a single trace, enabling debugging of silent failures, slow steps, and context loss in multi-agent workflows.

## Approach / What changed

Using Sentry’s Agent Tracing to capture AI agent traces, then applying five specific recipes: exporting conversations for evals via CLI, automating triage with Claude scheduled tasks via MCP, tracking costs with Vercel AI SDK, monitoring spend with dashboards and alerts, and sending OpenTelemetry spans to Sentry as an additional destination.

## Takeaways

- Agent Tracing captures model calls, tool executions, and handoffs in a single trace, allowing debugging of silent tool failures and slow steps in multi-step workflows.
- Production AI agent conversations can be exported into eval datasets using a single Sentry CLI command, with the first user message as input and last agent reply as output, along with cost and token metadata.
- AI agent error triage can be automated by having Claude’s scheduled tasks pull stats via the MCP server and file tickets in Linear when it finds broken tools, stuck loops, or costly runs.

## FAQ

### How can I turn production AI conversations into eval datasets?

Use the Sentry CLI to export production conversations with one command. The first user message becomes the input, the last agent reply becomes the output, and cost, tokens, and tool errors are included as metadata. You can then load these records into eval tools like Braintrust, Langfuse, promptfoo, or Arize Phoenix for scoring.

### How can I automate triage of AI agent errors?

Set up a scheduled task in Claude that runs daily and connects to Sentry through the MCP server. It pulls stats from the last 24 hours of conversations, reads a mix of failed and successful runs, and files tickets in Linear when it finds broken tools, stuck loops, or costly runs.

### How can I track LLM costs per conversation?

Use Sentry's Vercel AI SDK integration, which is enabled by default with the Sentry Next.js SDK. Name each AI SDK call and set a conversation ID to group turns. In Sentry's Explore → Agents view, you can see estimated cost by model, agent runs, and tool calls over time, plus individual conversation costs.

**Tags:** [Agent Tracing](https://yomu.fyi/topic/agent-tracing), [AI](https://yomu.fyi/topic/ai), [MCP](https://yomu.fyi/topic/mcp), [Observability](https://yomu.fyi/topic/observability), [Sentry](https://yomu.fyi/topic/sentry)

- Source: [Sentry](https://blog.sentry.io/agent-tracing-use-cases)
- Source URL: https://blog.sentry.io/agent-tracing-use-cases
- Ingested by Yomu: 2026-10-28T21:00:15.085Z

[Read original post](https://blog.sentry.io/agent-tracing-use-cases)
