Agent and Model Evaluations in Gemini Enterprise Agent Platform are now GA

Summary

Agent and model evaluations in Agent Platform are now generally available, providing one engine and consistent metrics for measuring agents and models during development and against live production traffic. The service offers more than 20 pre-built metrics across quality, safety, grounding, tool use, trajectories, and reference-based tasks, plus adaptive rubrics that generate case-specific criteria and a versioned, org-wide registry for custom code-based or LLM-as-a-judge metrics. Experiments can run locally or server-side, with Cloud Storage artifacts supporting auditability and reproducibility, while case generation, user simulation, and environment simulation expand coverage and model slow or failing backends. Online monitors grade sampled production traces, chart scores over time, and issue drift alerts through configured channels. Access is available through the SDK, REST API, agents-cli, console, and ADK; model-based metrics incur standard model-call charges, while code-based and computation metrics add no additional cost.

Takeaways

1. 1

   Adaptive rubrics generate case-specific pass/fail tests from eval case definitions, developer instructions, and tool declarations, then grade traces with a verdict and rationale.

2. 2

   Server-side experiments store every artifact in Cloud Storage, making runs auditable and reproducible; code-based and computation metrics add no additional cost.

3. 3

   Environment simulation can inject mocked data, forced errors, or added latency into tool calls, allowing agents to be tested against backend failures without touching production.