Evaluating AI agents: moving past the vibe check

Summary

Manual vibe checks for AI agents miss silent failures like retry loops, token waste, and hallucinated parameters while plausible final answers mask them. The post outlines a structured evaluation pipeline using the open-source agent-eval toolkit, starting with critical user journeys (CUJs) that define success in concrete, testable clauses. Synthetic user simulations (via ADK) generate multi-turn scenarios for initial trace collection, which developers grade with a ‘red pen’ before automating checks. Four layers of verification are recommended: unit tests, deterministic metrics, LLM-as-a-judge with boolean rubrics (avoiding unreliable 1-to-5 scales), and integration tests, all standardized on OpenInference/OpenTelemetry traces. The rollout progresses from Day 0 manual review through Day 1 automated checks to Day 2 CI/CD gates and shared dashboards, with real production traces fed back into the golden dataset.

Takeaways

1. 1

   AI agent failures are often invisible in final text output: retry loops, hallucinated tool parameters, and excessive token usage can occur while the agent returns a plausible answer. Evaluation must inspect intermediate traces (tool calls, reasoning steps) and operational metrics like latency and cost.

2. 2

   LLM judges are unreliable on qualitative 1-to-5 scales because the boundary between scores shifts between runs. Convert rubrics to boolean pass/fail questions, such as ‘Did the agent verify the account ID before calling the refund tool?’, paired with deterministic checks for tool parameters and reference comparisons.

3. 3

   To reuse evaluation across different agent frameworks (LangGraph, Google ADK, custom), decouple the pipeline from the orchestration layer by standardizing OpenTelemetry traces on OpenInference semantic conventions. The agent-eval toolkit supports this natively and routes traces to the Agent Evaluation service on the Gemini platform.

FAQ