Blocking Slow-Burn Attacks: Contextual Policies in Omnigent

Summary

Omnigent’s post examines how a vendor-review assistant can leak confidential pricing terms when an attacker hides an indirect prompt injection in a shared runbook. Because the malicious workflow is divided into ordinary actions, stateless checks approve each step even though the session as a whole is unsafe. The demonstration compares an unprotected run, which sends the summary externally, with a contextual policy that stores a running risk score, adds 30 for each document read, and denies email after the score exceeds 50. It also shows that agents cannot remove or disable policies, new policies require human approval, and any denial prevails when policies are combined. Runtime enforcement therefore preserves the block even when the agent has been misled.

Context

A vendor-review assistant follows a shared runbook that has been modified with an indirect prompt injection. The attack divides data exfiltration into ordinary steps, making each action appear safe when evaluated independently, even though the complete session sends confidential pricing terms to an external address.

Approach / What changed

Omnigent applies a stateful contextual policy that tracks session events with a running risk score. Each document read adds 30 points, and the email tool is denied once the score reaches the threshold of 50. Policies are enforced by the runtime; agents cannot remove or disable them, new policies require human approval, and a single denial prevails when policies are combined.

Takeaways

• The attack hides an outbound-data instruction in a shared runbook and divides the leak across document reads, summarization, and email, so no individual action appears dangerous.
• The demonstrated policy adds 30 risk points per document read; after two reads, the score reaches 60 and the subsequent send_report call is denied at the threshold of 50.
• Policy controls are tamper-resistant from the agent’s side: it has no removal or disable tool, additions require human approval, and a denial cannot be overridden by another policy.

Related reading