DatabricksNishith Sinha, Arun Pamulapati, Omar KhawajaProblem & solution
Innocent until combined: Blocking the lethal trifecta with Omnigent Contextual Policies
Summary
An AI agent can exfiltrate private data when one session combines access to confidential information, attacker-controlled content, and an external communication channel, even though each capability is individually authorized. Omnigent's contextual policy tracks those three legs as session state: private data, untrusted content, and exfiltration, with human-defined tool or argument-based classifications. It allows ordinary actions until both prerequisite legs are lit, then denies the outbound call that would complete the trifecta. In the support assistant example, a user-supplied ticket instructs the agent to read internal Q3 revenue figures and email them externally; without the policy, the confidential number leaves in a reply, while the protected run blocks that email. A password-reset ticket still proceeds because it lights only the untrusted-content leg, and an empty internal lookup does not light the private-data leg.
Context
The lethal trifecta arises when a session combines private-data access, untrusted content, and an outbound communication channel. Per-action authorization checks approve each capability separately and cannot detect that their sequence enables data exfiltration.
Approach / What changed
A human-defined Omnigent contextual policy records three session-state legs and denies an outbound call when private-data and untrusted-content legs are already lit. Leg assignments can be based on tools or inspected call arguments, and remain fixed during the session.
Takeaways
The policy blocks only the third leg after private-data and untrusted-content access have occurred; having both prerequisite legs lit does not itself stop the session.
A support ticket can be the untrusted input that directs an agent to read internal revenue data and send it to an external mailbox.
Leg classification is controlled by a human in agent configuration, preventing the agent from reclassifying data or content at runtime.
Source and freshness
- Source
- databricks.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.