DatabricksSam IngbarExplainer
Build durable agents with Temporal and Lakebase
Summary
Long-running personal-loan underwriting agents must preserve completed work, resume after worker failures, tolerate repeated tool calls, wait for reviewers, expose operational state, and retain an audit trail. The reference implementation combines Temporal durable Workflows and Activities with Lakebase Postgres, using Temporal Event History for replay and Lakebase schemas for run status, evidence, review state, messages, and metrics. Deterministic identifiers, constraints, guarded updates, and Postgres upserts make repeated Activity attempts target the same logical records, while Signals carry human decisions and long waits do not occupy workers. A Unity Catalog synced table supplies governed underwriting policy, and optional Change Data Feed publishes operational changes to Delta history tables. The implementation includes a React/FastAPI interface, mocked deterministic providers, OAuth credential refresh, and 21 passing tests, but does not validate production lending, compliance, security, scale, or an observed end-to-end feed.
Context
A cloud underwriting agent may outlive the request, worker, container, or deployment that started it. The system must recover completed work, retry operations without duplicating side effects, wait for human or external input without holding a worker open, expose current operational state, apply changing policy, and retain evidence and decisions for audit.
Approach / What changed
Temporal provides durable Workflow control flow, recorded Activity results, retries, timers, replay, and Signals for reviewer decisions. Lakebase Postgres stores queryable operational projections, while a Unity Catalog synced table supplies policy. Deterministic identifiers, database constraints, guarded updates, and upserts make repeated Activity attempts idempotent at the application level; Change Data Feed can publish operational history to Unity Catalog-managed Delta tables.
Takeaways
Temporal Event History lets a replacement worker replay recorded Activity results, timers, and Signals; an Activity whose completion was not recorded can be attempted again.
Lakebase stores application-facing run status, evidence, messages, review state, tool calls, events, and metrics separately from Temporal’s replay state.
The implementation does not include an observed end-to-end Change Data Feed run, and the feed requires enablement and destination verification in the target Databricks environment.
Source and freshness
- Source
- databricks.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.