Brandwatch AI: Speed Layer vs Trust Layer
Compare Brandwatch's AI speed features—Ask Iris, Query Writer, and AI summaries—with the governed checks that make social insights trustworthy before teams act.
Loading post…
Hugging FaceAntonio Tiene, Ander Alvarez Sanz, Oliver WirjadiProblem & solution
Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents
Summary
MCP agents can produce claims that are factually true but attributed to the wrong source, a failure known as cross-source conflation that source-blind checkers miss. ProvenanceGuard preserves source identity by decomposing answers into claims, routing each to its relevant tool output, and checking support and attribution separately, using local models like MiniLM and DeBERTa. On medical agent traces with 281 interactions, it caught 138 of 139 claims experts flagged, scoring 0.802 F1 on blocking decisions and correct source 86% of the time. It flagged 67 supported claims for review under a cautious policy and added roughly half a second per answer in local tests. Blocked answers can be fixed through a repair loop for rewriting or fallback, avoiding unverifiable responses while keeping sensitive traces in a controlled environment.
Context
The post addresses a gap in factuality verification for LLM agents that use the Model Context Protocol (MCP) to call multiple tools and sources. Existing source-blind checkers like RAGAS and MiniCheck pool all evidence and can pass claims that are true in one source but attributed to another (cross-source conflation), which is damaging in data-sensitive settings such as healthcare or customer support.
Approach / What changed
ProvenanceGuard is a post-generation verification layer that preserves source identity through the pipeline. It decomposes an answer into claims, routes each to its most relevant MCP tool output (using MiniLM), checks factual support with a DeBERTa NLI model, compares the supporting source to the source the answer names or implies, and emits per-claim source verdicts plus a global allow/block decision. Blocked answers can be repaired via a RARR-style loop. The reported setup uses local models but can be adapted to hosted services.
Takeaways
Existing source-blind checkers like MiniCheck and RAGAS pool all evidence and can miss cross-source conflation where a claim is true in the wrong source, a failure that can be as damaging as a wrong fact.
ProvenanceGuard achieves a 0.802 F1 on blocking decisions on medical agent traces, catching 138 of 139 claims experts flagged, while also identifying the correct source 86% of the time.
The system adds roughly half a second per answer and its repair loop resolved all blocked answers, though most ended in fallback text rather than substantive rewrites to avoid unverifiable responses.
FAQ
How does ProvenanceGuard prevent cross-source conflation?
It preserves source identity through the verification pipeline. Instead of pooling all evidence, it breaks the answer into claims, routes each to its relevant tool output, checks factual support, and verifies that the supporting source matches the source the answer names or implies.
What models does ProvenanceGuard use in its reported setup?
The reported setup uses MiniLM for routing claims to the relevant source, a DeBERTa NLI verifier model to check whether that source supports the claim, and a local language model to break answers into claims. These can be adapted to hosted models.
What overhead does ProvenanceGuard add per answer?
The system adds roughly half a second per answer on the reported local configuration. The NLI and routing calls themselves are in the tens of milliseconds, making it suitable as an offline verification gate.
Source and freshness
- Source
- huggingface.co
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Visual explainers
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.