How To Build Agents Users Can Trust

Summary

Ramp describes how it built and shipped LLM-backed agents for expense management, focusing on the trust risks created by low-quality or unexpected finance outputs. It recommends choosing ambiguous, high-volume problems with asymmetric upside, then combining reasoning and citations grounded in product or user-provided context with an explicit “Needs review” path when the agent is unsure. The policy agent uses expense policies to approve, reject, or escalate expenses, while deterministic rules such as dollar limits, vendor blocklists, and category restrictions constrain autonomy. Ramp reports that more than 65% of approvals are fully handled by the agent, while users can progressively move from suggestions to autonomous actions through workflow-based controls and evaluation.

Context

Finance workflows such as expense approval, merchant identification, and receipt parsing are tedious, high-volume tasks where simple heuristics do not always apply. Because unexpected or low-quality outputs can quickly erode trust in a finance product, the agents need to explain decisions, acknowledge uncertainty, defer to human judgment, and operate within user-defined limits.

Approach / What changed

Ramp combines LLM-based policy decisions with cited reasoning, predefined action categories, escalation when the agent is unsure, collaborative expense-policy editing, workflow-based autonomy controls, deterministic guardrails, staged adoption from suggestions to actions, and evals built from edge cases and reviewed golden datasets.

Takeaways

• The policy agent links its reasoning to relevant sections of the user’s expense policy, helping users verify decisions and helping developers identify prompt or context improvements.
• Instead of requesting numerical confidence scores, the system uses actionable categories: Approve, Reject, and Needs review; uncertain cases return to the established human escalation process.
• Users can define agent autonomy through workflow controls and hard stops, while dollar limits, vendor blocklists, and category restrictions remain deterministic guardrails.

Related reading