From Ringing Line to Resolved Contact: How a Call Center Works
Follow one contact through a call center: arrival, routing, handling by a human or AI agent, resolution, and the cost-per-contact math that decides which path to use.
Agent tracing records prompts, tool calls, and outputs as one span-linked run, then turns that record into eval datasets, alerts, and regression checks. Here is the five-stage flow from capture to decision.
Agent tracing is not just logging: it stitches each prompt, tool call, and output into a single span-linked run so you can see where an agent went quiet, slow, or lost context. The flow starts with instrumented capture, assembles one trace per run, and then reads failures off the timeline. From there, real conversations become eval datasets, and the same spans can be routed into an existing observability pipeline as an extra destination rather than a replacement. Engineers care because the trace that debugs a stuck loop can also catch a prompt regression that unit tests miss, as long as the trace stays accurate when a conversation is interrupted mid-stream.
Separate logs lose ordering and causality, so you cannot tell which tool call preceded a silent failure or a slow step. Capturing prompts, tool calls, and outputs in a single trace makes those relationships visible in multi-agent workflows. It also surfaces context loss, where an agent quietly drops information it was given earlier.
You export real production conversations and record the first user message as the input and the last agent reply as the expected output. Grading against that set gives you a repeatable baseline instead of ad hoc spot checks. Per-case diffs matter: in one Agents CLI run a swapped system prompt dropped the mean score from 4.0 to 2.75 and revealed tool misuse that unit tests never caught.
No. OpenTelemetry spans can be sent to an extra destination, so existing pipelines keep running unchanged while agent-specific views are added alongside them. The gotcha is trace fidelity: if an interrupted conversation is not truncated to what the caller actually heard, the exported eval data inherits that error as ground truth.
The posts behind it
Voice agents are now technically feasible because synthetic voices sound natural, speech-to-text is real-time, and agents can keep pace with human conversational timing. However, most pilots prove an agent can hold a conversation without proving it can be trusted with a real customer. The missing element is context: knowing who is calling, what is already in motion, and what the agent is allowed to do. Databricks provides this brain by combining customer context, an ontology of business relationships, and governance rules that already live on its platform. The agent uses fast decision models (System 1) for millisecond choices and large language models (System 2) for careful reasoning, with every decision recorded for traceability and improvement.
Sentry’s Agent Tracing captures prompts, tool calls, and outputs in a single trace, enabling debugging of silent failures, slow steps, and context loss in multi-agent workflows. The post details five recipes: exporting production conversations into eval datasets via a CLI command that records the first user message as input and the last agent reply as output. Another recipe automates error triage by having Claude’s scheduled tasks pull stats via the MCP server and file tickets in Linear for broken tools, stuck loops, or costly runs. Additional recipes cover tracking per-call costs with the Vercel AI SDK, building dashboards and alerts for spend monitoring, and sending OpenTelemetry spans to Sentry as an extra destination without replacing existing pipelines.
The Supabase integration for Make is now available, connecting Supabase projects to Make’s visual workflow builder. Users can query databases, read logs, deploy Edge Functions, and manage projects as workflow steps. The integration offers two connection methods: pre-built Supabase API apps for deterministic steps, and an MCP client for AI agents that get structured access to projects. Scenarios work with live data without requiring exports, eliminating the need for nightly syncs. Make tracks every scenario run and agent tool call, and each MCP tool is annotated as read-only or destructive for governance.
The Agents CLI eval namespace supports a hill-climbing quality loop through iterative generate, grade, compare, analyze, and optimize commands. Starting with a four-case dataset expanded from a single default, the first eval run produces a baseline mean score of 4.0 on the custom_response_quality metric with a standard deviation of 1.41. A regression introduced by a swapped chef persona system prompt dropped the mean to 2.75, and the per-case diff from eval compare revealed tool misuse on weather questions that unit tests could not catch. The workflow supports automated prompt optimization via eval optimize using GEPA, but the candidate prompts require human review to avoid overfitting to the metric. Effective hill-climbing requires a small dataset that runs on every change, snapshotting full result files for later regression analysis, and treating summary metrics with suspicion as shown by the stdev tightening from 1.41 to 0.96 during the regression.
Document extraction across dozens of PDF layouts is unreliable when fields are tuned in isolation; fixing one field can cause hidden regressions and apparent improvements may stem from normal run-to-run variation. The project extracting roughly 40 structured fields from partner-submitted order documents used human reviewers who corrected and approved extracted values, providing ground truth for evaluation. Field-specific rules go in JSON schema descriptions, document-level reasoning in a global prompt, precision mode handles complex structures, and citations diagnose failures. Candidates are versioned in MLflow Prompt Registry with aliases and evaluated against fixed ground truth using three runs to filter noise before promotion, outputting a BETTER, WORSE, or INCONCLUSIVE verdict. This approach reduced total errors by 32% and revision errors by 64%, creating a repeatable and regression-safe extraction workflow now automated by Genie Code.
More like this
Follow one contact through a call center: arrival, routing, handling by a human or AI agent, resolution, and the cost-per-contact math that decides which path to use.
A circuit breaker turns dependency failures into fast failures, then probes for recovery. See the state loop and how half-open probes can quietly consume worker capacity.
Cilium uses eBPF programs in the Linux kernel to enforce identity-based network policies and capture L7 observability for Kubernetes workloads. This diagram traces the path from packet arrival to policy verdict and Hubble flow.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.