Loading…
Topic
MLflow
37 posts about MLflow. Every summary links to the original.
- 37 posts
- 3 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Modern BSA/AML compliance on Databricks
AML operations are strained by fragmented systems, high false-positive volumes, manual case documentation, and opaque vendor scoring, leaving analysts focused on backlog rather than financial-crime intelligence. The proposed Databricks Data + AI Platform unifies transaction monitoring, KYC, sanctions, case history, and policy data under Unity Catalog, using Lakeflow Connect and a Bronze–Silver–Gold Delta architecture with masking, row-level security, and lineage. MLflow, Model Serving, Lakehouse Monitoring, and inference tables support institution-specific detection models, while Agent Bricks coordinates agents for evidence gathering, recommendations, and SAR drafting with analysts retaining final decisions. The architecture also uses Lakebase for governed operational state and Databricks Apps for analyst and executive experiences. Reported outcomes include a 75% reduction in false positives reaching the analyst queue and compressing three-to-six-hour investigations to minutes.
What is Explainable AI (XAI)?
Explainable AI (XAI) comprises techniques that help people understand how AI systems produce specific outputs, particularly when machine-learning and deep-learning models operate as black boxes. It distinguishes intrinsically interpretable models, such as decision trees and linear or logistic regression, from post-hoc methods including SHAP, LIME, counterfactuals, saliency maps and Grad-CAM. A typical workflow selects a model and prediction, applies a method suited to the model and audience, reviews outputs such as feature scores or heatmaps, and uses them to assess accuracy, fairness, reliability and compliance. The article stresses that post-hoc explanations are approximations rather than definitive proof, so teams should validate them with domain expertise and, where appropriate, combine methods; MLflow and Unity Catalog can preserve explanation artifacts, lineage and auditability.
Observability for any agent, anywhere: Production-ready tracing with OpenTelemetry & Unity Catalog on Databricks
Databricks supports writing OpenTelemetry (OTel) traces directly to Unity Catalog, where real-time telemetry is stored in Delta tables for governed analytics and retention. AI traces capture prompts, tool calls, responses, latency, and execution paths, enabling debugging, evaluation, and monitoring, while lakehouse storage also allows SQL queries, dashboards, joins with business data, and PII controls. The managed, serverless ingestion layer uses Zerobus Ingest to accept OTLP over gRPC from collectors and REST integrations, streaming spans, logs, and metrics to Unity Catalog without intermediate message buses. A LangGraph support manager assistant demonstrates instrumentation with mlflow.langchain.autolog() and an @MLflow.trace root span, while Genie is invoked through MCP for data-driven questions. The resulting traces can be searched in MLflow, evaluated at scale, and monitored continuously, with FAQ details stating 200 QPS starting throughput, no storage limit, and Unity Catalog governance options for access control, masking, and filtering.
From "What Happened?" to "What Will Happen?"
Databricks Genie makes descriptive analytics accessible in natural language, but predictive questions still require specialized data science workflows and carefully prepared datasets. This post presents a multi-agent supervisor deployed as a Databricks App, combining Genie, TabPFN, and Agent Bricks to turn business questions into predictions. The orchestrator asks Genie to use governed Lakehouse data, schemas, relationships, and semantics to generate labeled training data through SQL, then sends it to TabPFN, which predicts in a single forward pass without feature preprocessing, model selection, or hyperparameter tuning. The resulting conversational experience supports descriptive and predictive analytics with Unity Catalog lineage and access control, while an MLflow GenAI evaluation harness monitors reliability and regressions. Its central limitation is that predictions depend on Genie producing a meaningful dataset with a clear label, so missing signals, joins, outcomes, or agent omissions can make results unreliable.
How to Build Real-Time Fraud Detection using Spark Real-Time Mode and Lakebase
Card fraud operates in seconds, while organizations need to score and block suspicious transactions before settlement without adding a separate streaming stack. This Solution Accelerator presents an open-source, end-to-end implementation on Databricks, covering transaction ingestion, Spark Real-Time Mode processing, fraud scoring, Lakebase feature serving, and a Databricks Apps monitoring dashboard. RTM processes streaming workloads in milliseconds and targets sub-300ms latency, while Lakebase supplies a managed, serverless Postgres serving layer with sub-millisecond reads. The design keeps batch ETL, training, streaming, and governance within the same platform. Deployment progresses from a synthetic rate-source quick start to Kafka-based parsing, stateful per-card velocity tracking with TTL, enrichment, weighted scoring, routing, and optional MLflow-trained RandomForest scoring, followed by a Streamlit app for live decisions.
Databricks context engineer associate: the industry’s first certification for reliable AI agent systems
Databricks announces the Databricks Certified Context Engineer Associate, described as the industry’s first certification purpose-built for context-aware AI systems. It assesses whether practitioners can design, assemble, and govern the information AI agents receive at inference time, including system prompts, retrieval, memory, external tools, context-window management, and governance. Candidates work with Databricks technologies such as AI Search, Lakebase, MLflow, MCP, and Unity Catalog, while advanced coverage includes multi-agent systems, long-horizon workflows, and evaluation of context decisions. The beta exam will be available free to Data + AI Summit attendees, who may take it once; beta results are expected in 6–8 weeks.
The Rosetta stone of CPS: Claroty’s AI-powered library
CPS asset identification is difficult because 88% of CPS assets do not transmit an exact product code and 76% use product codes that differ from the vendor's official records. Claroty's AI-Powered CPS Library addresses this entity-resolution problem with a hybrid architecture that combines classic matching, statistical inference, generative AI, NLP, reasoning agents, and human-in-the-loop review to reconcile protocol data, vendor records, manuals, and vulnerability information. Its Databricks-based lakehouse uses Delta Lake, Unity Catalog, Spark pipelines, Model Serving, MLflow evaluations, Lakebase, and Databricks Apps to maintain governed, auditable mappings across a catalog of more than 17 million assets. The post reports 25% improved vulnerability-attribution accuracy and says early tests gave 56% of analyzed devices new or updated recommendations for outdated firmware, while low-confidence mappings remain subject to expert review.
Clinical operations intelligence belongs on the Lakehouse
The Site Feasibility Workbench addresses fragmented clinical operations data by placing a site-selection decision-support application, its models, and data on Databricks. Released as a fully open-source Databricks App, it combines a FastAPI backend, React frontend, Unity Catalog-governed data, Lakebase operational state, and AI/BI Genie within the workspace. TA-segmented LightGBM models use the organization’s CTMS, EDC, and IRT history alongside real-world evidence, patient access, qualification, Open Payments, and protocol-execution signals to rank sites. Each prediction stores SHAP attributions in a governed Unity Catalog Delta table, while MLflow tracks training runs and model versions; saved shortlists persist to Lakebase and Genie answers governed-data questions. The six-step workflow supports protocol selection through final shortlisting, including diversity considerations; the release is a decision-support layer rather than a source of record, and deployment reportedly takes about 30 minutes before security review and validation.
ML based ranking using Nrtsearch
Nrtsearch’s Inference Plugin embeds machine-learning ranking in the Lucene-based search engine, addressing scalability problems in a two-stage design where a separate inference service scored candidates. Previously, large candidate sets and feature payloads created network-transfer and serialization overhead; the new flow extracts features from index documents or request parameters and scores documents within replica nodes. An ML Scorer selects models, transforms features, runs MLeap-based inference, and aggregates scores, with support for XGBoost, neural networks, multi-model execution, batch inference, and custom Java scorers. Models are loaded from MLflow through cluster configuration, while teams can use Lucene expressions or implement modular interfaces; testing, canary deployment, rolling updates, and Prometheus metrics support operations, and future work includes GPU-based ranking and additional inference engines.
Using MemAlign to Improve Evaluation of Traditional Machine Learning in Genie Code
Genie Code’s traditional machine learning workflows are difficult to evaluate because notebook quality depends on code, best practices, and adaptation to data and task. The team built an evaluation pipeline with nine LLM judges, each scoring applicable dimensions—including data exploration, imputation, training, MLflow logging, and cell organization—from 1 to 3, using test cases across classification, regression, and forecasting. On 50 human-graded notebooks, initial judge alignment was weak in several areas, including model training (MAE 0.680) and model use (0.562), with positivity bias contributing to missed technical flaws. Applying MLflow’s MemAlign, which combines generalized semantic guidelines with retrieved episodic examples, reduced judge error by 74–89% on the most important dimensions; removing episodic memory weakened most gains, while data exploration showed a nonsignificant regression.
Built in, not bolted on: What AI-native actually means in cybersecurity
The discussion defines AI-native cybersecurity applications as systems architected with intelligence at their core, rather than traditional products with AI added later. It links tool sprawl to slower threat detection and response and argues that proprietary, context-rich security telemetry is the foundation for adaptive defense. Barracuda uses Databricks to consolidate fragmented data, normalize schemas, support real-time streaming detection, run ML operations through MLflow, and power natural-language log search across billions of security events with strict data isolation. The work began by defining customer outcomes, then progressed through incremental delivery; normalized data enabled models and agents to use cross-domain context. This approach extended across WAF-as-a-service, automated configuration, API security, and bot protection while shared outcomes aligned product, data science, engineering, and business teams.
Model risk management in 2026: A banker's guide to the revised interagency guidance
The post presents the April 17, 2026 revision to model risk management guidance as a shift toward risk-based, principles-driven oversight, replacing several earlier issuances from the Federal Reserve, FDIC, and OCC. It identifies five operational changes: materiality tiering, end-to-end lifecycle governance, reproducible effective challenge, continuous drift and performance monitoring, and application of MRM principles to GenAI and agentic systems. Its proposed response is a Databricks reference architecture built on Unity Catalog, Delta Lake, Lakeflow Declarative Pipelines, Feature Store, MLflow, Model Registry, Model Serving, and assurance tools. The design makes tiering metadata-driven and turns lineage, validation, approvals, monitoring, documentation, and retirement records into evidence generated during normal model work. The stated conclusion is that a unified substrate can reduce integration and manual evidence work, allowing regulatory changes to become configuration exercises rather than multi-quarter programs.
AI App Development: Guide To Building AI-Powered Apps
Production-grade AI app development combines model orchestration, transactional state, governance, security, and live-data integration rather than only interface construction. The guide recommends defining users, outcomes, success metrics, timelines, and AI-relevant journeys, then shipping must-have features before enhancements. It evaluates builders by end-to-end support, technical audience, platform integration, code export, CI/CD, data connections, pricing, and deployment limits. For Databricks teams, it describes Databricks Apps, Declarative Automation Bundles, and Lakebase: serverless app compute, version-controlled deployment, and managed PostgreSQL with synced Unity Catalog Delta tables. It also recommends monitoring outputs, collecting correction feedback, refining prompts with automated evaluations, and governing models through MLflow, concluding that integrated infrastructure can leave teams focused on production AI workflows.
Get hands-on with agents, vibe coding and more at Data+ AI Summit
Data + AI Summit returns to San Francisco from June 14 through June 18, 2026, with Databricks Training and Certification offering more than 20 hands-on courses across AI agents, vibe coding, data engineering, and Lakebase. Training-exclusive days on Sunday and Monday are designed to provide dedicated learning time, while sessions continue during the conference week. New courses cover agentic development with Genie Code, Claude Code, Cursor, MCP servers, and structured prompting; production agent deployment with MLflow’s ResponsesAgent framework and tracing; Lakeflow Spark Declarative Pipelines; and AI/BI dashboards and Genie Spaces. Onsite certification exams cost $100 plus taxes, a 50% discount, and seven listed exams are available. The Learning Hub adds labs, skills assessments, customized learning plans, and community access.
Governing coding agent sprawl with Unity AI Gateway
Databricks introduces coding agent support in Unity AI Gateway to address security, cost, and visibility challenges created by organizations using multiple coding tools. The gateway provides a unified governance hub for Codex, Cursor, and Gemini CLI, combining access controls, usage statistics, operational observability, cost management, guardrails, and inference capacity. Agent access can be governed centrally, with audit logs in Unity Catalog, MCP servers managed in Databricks, MLflow tracing, shared cost limits, and coding metrics and traces ingested into Unity Catalog-managed Delta tables through OpenTelemetry. The announcement says developers can authenticate with Databricks credentials across connected services, use centralized budgets and model capacity, and let organizations analyze adoption, developer velocity, and rate-limit pressure in the lakehouse; support for Cursor, Gemini CLI, and Codex CLI is available to all Databricks customers.
Banks don't have an AI problem — they have a data platform problem
The post argues that banks’ difficulty scaling AI stems less from model capability than from fragmented data, weak governance, and limited real-time access. Examples from CBA Live 2026 include model drift in credit scoring, fragmented customer signals, and a collections model that predicted, with 85% accuracy, how many days newly delinquent accounts would take to cure using governed data. It also presents Erica’s 3.2 billion interactions since 2018 as evidence that production AI requires continuous tuning, monitoring, and edge-case management, while generic models decay in frontline settings. The proposed Databricks architecture combines Lakehouse, Unity Catalog, Lakeflow, Lakebase, MLflow, Model Monitoring, Online Feature Store, Genie, and Agent Bricks to support governed analytics, low-latency decisions, auditable models, and constrained agent actions. Its conclusion is that a shared data platform should precede additional AI use cases because it improves deployment speed, trust, explainability, and regulatory defensibility.
Agent Bricks: The governed enterprise agent platform
Agent Bricks is presented as Databricks’ enterprise platform for building, deploying, and governing agents that operate on business data under real identities, permissions, and operational constraints. The platform combines multi-model and framework support, execution, routing, fallback, cost optimization, and unified governance through Unity Catalog and AI Gateway, including on-behalf-of token passing and observability across data, models, MCPs, and APIs. Its context layer uses metadata such as schemas, business definitions, lineage, permissions, and data-quality signals, while Genie Spaces, Document Intelligence, Knowledge Assistant, and Agent Mode address structured and unstructured business information. The announcement includes general availability for Document Intelligence, Custom Agents on Apps, and Supervisor Agent, plus AI Gateway guardrails, managed OAuth MCP Connectors, web search, and MLflow’s CLEARS evaluation framework; the post reports 70% higher accuracy than standard RAG and a 30% improvement in multi-step workflows.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.