Audit Is a Provenance Chain, Not a Log File
An audit is a provenance chain: what was stored, where each fact came from, and what was used for each decision. This explainer maps the three links that keep agent behavior traceable and fixable.
TwilioJesse SumrakTutorial
To audit what an AI agent knows about a customer, three questions must be answered: what the agent stored, where facts came from, and what it used for each answer. Twilio Conversation Memory stores customer knowledge as traits, observations, and conversation summaries, each with provenance including source, conversation IDs, and revision history. Agent Connect passes retrieved memory to the developer’s code on every message, enabling logging of which memories were used per response, including relevance scores. The guide also covers correcting or deleting incorrect memories and building a persistent audit trail by exporting Memory events via Twilio Event Streams, tied together with conversation IDs. This approach gives human agents and compliance teams the ability to trace, explain, and fix agent behavior, with Agent Copilot displaying knowledge sources.
How to audit what an AI agent knows about a customer using Twilio Conversation Memory, covering provenance, retrieval logging, correction, and audit trail building.
Use Twilio Conversation Memory’s API to review traits, observations, and summaries; trace each observation to its source conversation via provenance fields; log retrieved memories from Agent Connect in your code; correct or delete incorrect memories; and export Memory events via Event Streams to build a persistent audit trail.
Twilio Conversation Memory stores customer knowledge as traits, observations, and conversation summaries, each with provenance including source and conversation IDs.
Agent Connect passes retrieved memory to your code on every message, allowing you to log exactly which memories were used per response.
You can correct or delete individual observations, traits, or summaries via the Memory API; deletions are irreversible and asynchronous.
Twilio Conversation Memory records each observation's source and the IDs of the conversations it came from. Agent Connect passes retrieved memory to your code on every message, so you can log what the agent used for each response.
Twilio Conversation Memory lets you list a customer's traits, observations, and conversation summaries through its API or in the Console, with a revision history showing how each observation changed.
Twilio Conversation Memory lets you update or delete individual observations, traits, and summaries through its API, or delete a customer's profile with its identifiers and traits. Deletions are irreversible and process asynchronously.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
An audit is a provenance chain: what was stored, where each fact came from, and what was used for each decision. This explainer maps the three links that keep agent behavior traceable and fixable.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.