GoogleNakul JavaliBenchmark
How Agent Anomaly Detection scores on benchmarks we didn't write
Summary
Agent Anomaly Detection, a new security feature on Gemini Enterprise Agent Platform, flags sessions where an AI agent acts outside its intended boundaries by reading traces and logs asynchronously. Google tested it against five public, non-Google-written benchmarks for agent goal hijack, the top OWASP risk for agentic applications. The system flagged 87% of harmful instructions that the agent carried out, with false alarms on 2% to 16% of benign sessions depending on the agent’s tools. It performed weakest on poisoned memory attacks, catching only 4 of 18 such sessions. The study used one agent model and simulated apps, so production false-alarm rates may differ.
Context
AI agents now read email, book travel, and move money, making them vulnerable to indirect prompt injection where harmful instructions arrive inside data returned by a tool. OWASP ranks this agent goal hijack first in its Top 10 for Agentic Applications (2026). Google announced Agent Anomaly Detection in Private Preview on Gemini Enterprise Agent Platform in September 2026.
Approach / What changed
Agent Anomaly Detection reads agents’ traces and logs asynchronously, without adding latency, and flags sessions where an agent acts outside its intended boundaries. It was tested against five public, non-Google-written benchmarks (AgentDojo, InjecAgent, ASB, AgentDyn, AgentHarm) using an agent built with the Agent Development Kit (ADK) and gemini-2.5-flash on simulated apps. The agent first ran at least 300 benign sessions so the system could learn normal behavior. The benchmark’s own deterministic Python check labeled each session by reading the final state of the apps, preventing injected instructions from misleading the grader.
Takeaways
Agent Anomaly Detection flagged 87% of harmful instructions that the agent carried out across four public benchmarks, with the highest rate (99%) on AgentDyn and the lowest (81%) on AgentHarm.
False alarms ranged from 2% to 16% of benign sessions; agents using high-impact tools like social media posting and terminal commands triggered more false positives.
Poisoned memory is the clear gap: Agent Anomaly Detection caught only 4 of 18 sessions where instructions planted in long-term memory made the agent act.
FAQ
What percentage of carried-out harmful instructions did Agent Anomaly Detection flag?
Agent Anomaly Detection flagged 87% of the harmful instructions that the agent carried out. It flagged 85% on AgentDojo, 87% on InjecAgent, 99% on AgentDyn, and 81% on AgentHarm.
How many false alarms did Agent Anomaly Detection raise?
False alarms ranged from 2% to 16% of benign sessions, depending on the tools the agent uses. On injection benchmarks with office tasks, it flagged 2% to 4%. On AgentHarm, where benign tasks include posting to social media and running terminal commands, it flagged 16%.
How well did Agent Anomaly Detection detect poisoned memory attacks?
Agent Anomaly Detection caught only 4 of 18 sessions where instructions planted in the agent's long-term memory made it act. This is the clear gap identified in the study.
Source and freshness
- Source
- developers.googleblog.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.