Loading…
Modern BSA/AML compliance on Databricks
Kateryna Savchyn, Pavithra Rao, Mimi Park, Emerson Bayuk
- Source
- Databricks
- Published
- Added to Yomu
Summary
AML operations are strained by fragmented systems, high false-positive volumes, manual case documentation, and opaque vendor scoring, leaving analysts focused on backlog rather than financial-crime intelligence. The proposed Databricks Data + AI Platform unifies transaction monitoring, KYC, sanctions, case history, and policy data under Unity Catalog, using Lakeflow Connect and a Bronze–Silver–Gold Delta architecture with masking, row-level security, and lineage. MLflow, Model Serving, Lakehouse Monitoring, and inference tables support institution-specific detection models, while Agent Bricks coordinates agents for evidence gathering, recommendations, and SAR drafting with analysts retaining final decisions. The architecture also uses Lakebase for governed operational state and Databricks Apps for analyst and executive experiences. Reported outcomes include a 75% reduction in false positives reaching the analyst queue and compressing three-to-six-hour investigations to minutes.
Context
AML teams face productivity constraints from more than 10 siloed systems, false-positive rates estimated at 90–95% of transaction-monitoring alerts, manual evidence and SAR documentation, and opaque vendor scoring that complicates model risk management and regulatory explanations.
Approach / What changed
The proposed architecture combines governed lakehouse data, Unity Catalog, Lakeflow Connect, MLflow-based detection models, Model Serving, Lakehouse Monitoring, Agent Bricks, Vector Search, entity resolution, Lakebase, and Databricks Apps. Its five layers can be adopted independently or as a complete stack, with analysts retaining final case decisions.
Takeaways
- Unity Catalog provides masking, row-level security, quality controls, and lineage from source transactions through risk scores, evidence chains, and filed SARs.
- MLflow champion/challenger workflows, Model Serving, Lakehouse Monitoring, and inference tables support institution-specific model deployment, drift observation, and retraining from analyst feedback.
- Agent Bricks coordinates investigation agents and SAR drafting, while human analysts decide whether to escalate, dismiss a false positive, or file a SAR.