---
title: "Databricks"
description: "Data and AI platform for data engineering, analytics, machine learning, and generative AI."
---

# Databricks
> Data and AI platform for data engineering, analytics, machine learning, and generative AI.

## Articles

### [The turbine that tried to tell you it was failing](https://yomu.fyi/post/the-turbine-that-tried-to-tell-you-it-was-failing.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Caitlin Gordon
- Published: Apr 30, 2026

Gas turbines produce millions of daily readings—vibration, temperature, pressure, flow rates, and electrical output—but maintenance teams may learn about warning signals only when an unplanned outage occurs. The post argues that predictive maintenance has struggled less because of model capability than because decision-makers lack fluid access to model findings and operational context. Databricks Genie provides a conversational interface to sensor, maintenance, dispatch, cost, and regulatory data, allowing questions about elevated vibration against maintenance baselines and timing maintenance around outage cycles. It is presented as decision support, not automation, with time-series analysis, maintenance history, integrated costs, and compliance context helping asset managers investigate fleets and act with more confidence.


### [Predicting readmissions isn't enough. Acting in time is.](https://yomu.fyi/post/predicting-readmissions-isn-t-enough-acting-in-time-is.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Adam Crown
- Published: Apr 30, 2026

Readmission risk models can identify patients likely to return within 30 days, but prediction alone does not ensure timely intervention. In large health systems, risk information may remain in population-health dashboards or EHR flags without reaching the care coordinator with enough context to create an effective post-discharge plan. The post presents Databricks Genie as a natural-language interface for governed clinical and outcomes data, allowing leaders to query readmission rates and compare periods while combining EHR, operational, financial, risk-score, intervention, and outcome information. Genie operates within Unity Catalog governance, with access controls, audit logging, and de-identification policies, and is described as supporting EHR integration and clinical taxonomy awareness; the intended result is to shorten the path from prediction to action.


### [Clinical trials run longer than they have to. That's a patient problem.](https://yomu.fyi/post/clinical-trials-run-longer-than-they-have-to-that-s-a-patient-problem.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Adam Crown
- Published: Apr 30, 2026

Clinical trial operations can lose weeks because site-level performance data is reviewed with a 2–4 week lag: enrollment velocity is monthly, while protocol deviation rates are assessed quarterly. The post presents Databricks Genie as a natural-language interface to unified trial data, allowing clinical operations leaders to query enrollment, screen-failure, protocol-deviation, query-response, and data-entry metrics across sites. Its stated capabilities include automatic site comparison, integration across CTMS, EDC, safety databases, and site-performance data, plus protocol-aware reasoning and traceability to source records for GCP documentation. The example query identifies Phase II oncology sites with screen-failure rates above 40% over 60 days and compares enrollment pace with activation targets, positioning earlier detection as a way to reduce timeline impact and speed treatment access.


### [Network quality is a revenue problem, not a technical one](https://yomu.fyi/post/network-quality-is-a-revenue-problem-not-a-technical-one.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Elena Tesser
- Published: Apr 30, 2026

Telecommunications network operations centers monitor extensive performance telemetry, but the source describes a gap between technical network events and the customers, contracts, and revenue they affect. A degraded tower serving 12,000 postpaid customers averaging $85 per month presents a different business problem from one affecting 12,000 prepaid customers with high churn propensity, while enterprise SLA exposure can remain hidden from the NOC. Databricks Genie connects network elements, serving areas, customer records, contract terms, SLA thresholds, and churn-risk scores so leaders can query commercial impact in seconds. It also supports proactive alerts based on defined commercial thresholds, enabling restoration and investment decisions to account for customer impact rather than technical severity alone.


### [Shelf availability starts with better demand visibility](https://yomu.fyi/post/shelf-availability-starts-with-better-demand-visibility.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Sarah Duffy
- Published: Apr 30, 2026

Retail out-of-stock rates in grocery and general merchandise typically run between 7% and 10%, leaving roughly one in ten sought-after items unavailable at a given moment. The revenue impact is real, but repeated shelf gaps can cause customers to build shopping habits elsewhere. Modern retail replenishment requires real-time synthesis of POS velocity by store and SKU, distribution-center inventory, on-order quantities and delivery windows, supplier fill-rate histories, promotional calendars, and other demand signals. Databricks Genie is presented as a conversational interface to the full inventory and demand environment, allowing leaders to ask which high-velocity SKUs are projected to stock out within 72 hours and receive current on-order positions in seconds rather than hours. Its stated differentiators include store-SKU granularity, shared promotional, event, and weather data, supplier performance context, and cross-distribution-center rebalancing; the post says Genie is available today.


### [When predicting the next hit requires more than intuition](https://yomu.fyi/post/when-predicting-the-next-hit-requires-more-than-intuition.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Elena Tesser
- Published: Apr 30, 2026

Content investment decisions in entertainment commit substantial funds despite limited information, because leaders often combine executive instinct, competitive benchmarks, and historical performance data. Although companies hold viewing and subscriber signals such as episode completion, skip patterns, genre performance by demographic, acquisition, and retention, those insights are frequently buried in dashboards and spreadsheets. The post presents Databricks Genie as a natural-language interface that lets content leaders query governed performance data directly, including comparisons linking content types with 90-day retention, subscriber demographics, lifetime value, acquisition source, and historical greenlights. It argues that current, conversational access can complement creative judgment by reducing dependence on delayed analyst recommendations, while external market data can add competitive context.


### [Approximate answers, exact decisions: New sketch functions for analytics](https://yomu.fyi/post/approximate-answers-exact-decisions-new-sketch-functions-for-analytics.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Daniel Tenedorio, Kent Marten, Gengliang Wang, Chenhao Li
- Published: Apr 29, 2026

Databricks now supports four sketch function families, built on Apache DataSketches, for replacing expensive exact percentiles, distinct counts, set operations, top-K rankings, and metric aggregations with bounded-memory approximations. KLL sketches answer quantiles, Theta sketches support unions, intersections, and differences, approximate top-K sketches track frequent items, and Tuple sketches combine distinct counting with aggregated metrics such as customer revenue. The sketches can be built during ETL, stored as columns in Delta tables, and merged on read, allowing dashboards and streaming pipelines to avoid rescanning raw data; SQL, DataFrame, and Structured Streaming pipelines are supported. The stated trade-off is configurable 1–2% relative error, with the source citing a 1000x speedup for applicable workloads. Exact methods remain appropriate for financial auditing, compliance reporting, and other precision-required uses.


### [Companies winning with AI built the data layer first](https://yomu.fyi/post/companies-winning-with-ai-built-the-data-layer-first.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Aly McGue
- Published: Apr 29, 2026

Trinity Industries’ experience is presented as a case for treating the data layer, rather than models or dashboards, as the foundation of enterprise AI. The railcar manufacturer migrated 95% of its enterprise data to a single Databricks lakehouse, adopted Medallion architecture, moved transformations upstream, and retired legacy dashboards containing nearly 600 measures. That consolidation supports streaming ETA predictions, procurement agents, and Genie conversational analytics: the ETA model is reported as 50% more accurate than industry ETAs, agents helped increase on-time material delivery by 15%, and Genie handles more than 1,000 questions monthly. The migration took close to a year, followed by six to eight months of additional work, but the account argues that trusted, standardized data enables experimentation, automation, and broader employee access to analysis.


### [Rethinking SQL ETL for modern data platforms](https://yomu.fyi/post/rethinking-sql-etl-for-modern-data-platforms.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Matt Jones, Shanelle Roman
- Published: Apr 29, 2026

SQL ETL is often distributed across a warehouse, transformation framework, orchestrator, monitoring, lineage, and data-quality systems, creating operational fragmentation as data teams scale. The post proposes a unified platform model that combines execution, orchestration, observability, governance, and optimization while supporting analytics engineers, warehouse engineers, and analysts through dbt, scripts and stored procedures, Materialized Views, declarative pipelines, and no-code tools. Databricks implements this model with shared execution, governance, and observability, using serverless infrastructure and AI-driven optimization alongside open table formats and ANSI SQL. The stated benefits include reduced operational overhead, support for batch and real-time workloads, and incremental modernization rather than disruptive rewrites. Reported examples include HP’s 32% cloud savings and 36% lower combined job runtime after moving to serverless compute, and Adobe reducing query execution from eight minutes to three seconds with Materialized Views.


### [Stripe data now available on Databricks via Databricks Marketplace](https://yomu.fyi/post/stripe-data-now-available-on-databricks-via-databricks-marketplace.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Justin Fenton, Harish Gaur, Matt Napoli
- Published: Apr 29, 2026

Stripe Data Pipeline is now available on Databricks Marketplace through Delta Sharing, allowing payment and business data to be analyzed in a Databricks workspace without ETL. Stripe data—including transaction records, customer histories, subscriptions, refunds, and payouts—remains in Stripe’s infrastructure and is queried directly through Unity Catalog, where it can be joined with other tables. The approach replaces polling integrations and custom jobs, avoiding per-call charges, connector licensing fees, data duplication, and maintenance code while providing row- and column-level access controls, audit trails, and compliance features. The shared data is positioned for AI-native uses such as continuous transaction monitoring, churn scoring, LLM-powered retention workflows, Genie queries, and finance analytics apps. Getting started involves visiting the Stripe Data Pipeline listing on Databricks Marketplace.


### [Databricks and Stripe projects: Infrastructure built for agents](https://yomu.fyi/post/databricks-and-stripe-projects-infrastructure-built-for-agents.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Brad Van Vugt, Guillaume Rivals
- Published: Apr 29, 2026

The post announces Databricks as a launch partner for Stripe Projects, an agent-first CLI that lets AI agents discover, provision, and pay for Neon databases without human interaction. It addresses the manual work still required to provision services, navigate interfaces, configure accounts, and enter payment details, which can slow autonomous app development. With Lakebase architecture, Neon can provide production-ready Postgres in under 350ms, while decoupled compute and storage support rapid creation and teardown of OLTP databases. Serverless scaling adjusts compute to traffic and can scale to zero, while zero-copy branching creates isolated production-data branches for testing, migrations, and prompt experiments without risking the primary environment. The post also notes Databricks’ Stripe Data Pipeline is available in the Databricks Marketplace for analyzing Stripe data without ETL.


### [Agents are ready, but your architecture probably isn't](https://yomu.fyi/post/agents-are-ready-but-your-architecture-probably-isn-t.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Catherine Brown
- Published: Apr 29, 2026

Enterprise AI initiatives often produce activity rather than value because organizations start with technology instead of a defined outcome and overlook data architecture, governance, and semantic context. Agentic systems add risk when they can send messages, update records, place orders, or delete records, making permissions and situational controls essential. The discussion argues that dashboards and batch pipelines are poorly matched to low-latency, high-scale agent workloads, which require transactional infrastructure alongside existing analytics. Lakebase is presented as that transactional foundation, while AgentBricks, Databricks Apps, and Genie provide agent development and monitoring, application delivery, and conversational data access. The recommended path is to define success first, isolate a focused pilot, learn what works, and redesign underlying processes rather than merely add AI to them.


### [Interoperability between Unity Catalog and Google BigQuery via catalog federation](https://yomu.fyi/post/interoperability-between-unity-catalog-and-google-bigquery-via-catalog.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: John Spencer, Jason Reid
- Published: Apr 29, 2026

Databricks and Google Cloud announce bidirectional catalog federation that lets customers access the same data from Unity Catalog or BigQuery without duplication. The interoperability uses Apache Iceberg REST catalog APIs, enabling both platforms to read and write the same tables through a common open interface. Google Cloud's catalog federation is in preview for reading Unity Catalog-managed tables from BigQuery, while Google Cloud's Lakehouse federation is in private preview for Databricks users to govern and read foreign Iceberg tables. Unity Catalog's policies, fine-grained access controls, and lineage tracking apply to federated data, and users can query it with Genie using natural language. The announcement positions open standards as a path to interoperability across data platforms and previews future catalog metadata exchange through Iceberg REST catalog APIs.


### [Built in, not bolted on: What AI-native actually means in cybersecurity](https://yomu.fyi/post/built-in-not-bolted-on-what-ai-native-actually-means-in-cybersecurity.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Aly McGue
- Published: Apr 28, 2026

The discussion defines AI-native cybersecurity applications as systems architected with intelligence at their core, rather than traditional products with AI added later. It links tool sprawl to slower threat detection and response and argues that proprietary, context-rich security telemetry is the foundation for adaptive defense. Barracuda uses Databricks to consolidate fragmented data, normalize schemas, support real-time streaming detection, run ML operations through MLflow, and power natural-language log search across billions of security events with strict data isolation. The work began by defining customer outcomes, then progressed through incremental delivery; normalized data enabled models and agents to use cross-domain context. This approach extended across WAF-as-a-service, automated configuration, API security, and bot protection while shared outcomes aligned product, data science, engineering, and business teams.


### [Operationalizing AI for public sector fraud prevention](https://yomu.fyi/post/operationalizing-ai-for-public-sector-fraud-prevention.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Johnathan Tafoya, Kacey Hertan
- Published: Apr 28, 2026

Public-sector agencies face fraud methods including synthetic identities, deepfake-enhanced documentation, and personalized social engineering, while legacy controls remain fragmented and difficult to scale. The fictional Services Bureau demonstrates an operating model that combines Databricks Apps, Lakebase, Unity Catalog, Delta Sharing, Agent Bricks, and AI/BI Genie in a single fraud-operations environment. Governed data lands in Delta tables; Unity Catalog applies attribute-based access control, masks PII by role, and provides lineage, while agents connect live lakehouse queries, agency policies, and external fraud signals through MCP. Analysts review evidence and recommendations, then approve, override, or escalate cases, keeping human judgment central. The described workflow turns weeks of manual investigation into a day, supports dashboards and conversational SQL, and is presented as making fraud decisions faster, more secure, transparent, and defensible.


### [From months to minutes: Building real-time clinical data pipelines with natural language](https://yomu.fyi/post/from-months-to-minutes-building-real-time-clinical-data-pipelines-with.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Matthew Giglia, Tim Kessler, Assunta Carey-Saylor
- Published: Apr 28, 2026

Healthcare data teams often spend months integrating EHR systems, normalizing HL7, CCD, and X12, and routing data through intermediary storage before analytics, creating latency and maintenance burden. Databricks and Redox describe a pipeline model that combines Redox MCP Server, natural-language prompts, and Databricks Zerobus Ingest to build integrations inside Databricks and stream clinical data directly into Unity Catalog managed tables. The post says Zerobus provides subsecond latency, while the MCP Server identifies environments, suggests workflow steps, executes integration tasks, and surfaces validation signals such as logs and performance summaries. A demonstrated workflow retrieved a recent patient admission as structured data with a plain-language summary, and Redox writeback can return AI outputs to EHRs for point-of-care action. The same foundation is presented as enabling real-time use cases and Redox Agents built with Databricks Genie Spaces.


### [Agentic data engineering with Genie Code and Lakeflow](https://yomu.fyi/post/agentic-data-engineering-with-genie-code-and-lakeflow.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Gal Oshri, Camiel Steenstra, Lennart Kats, Joanna Zouhour
- Published: Apr 28, 2026

Genie Code is presented as a natural-language assistant for developing, orchestrating, deploying, and debugging data pipelines and jobs. It uses pipeline and job context, including code, configuration, run results, Unity Catalog metadata, lineage, popularity, and code samples, to help engineers discover datasets and understand data flows. Engineers can describe pipelines or jobs, and Genie Code can generate Spark Declarative Pipelines with Bronze, Silver, and Gold layers, sources, transformations, data quality expectations, and outputs, then configure orchestration, schedules, dependencies, Auto Loader, AutoCDC flows, and Declarative Automation Bundles. It also analyzes failures and unexpected row-count or schema changes, proposes cross-file updates with reviewable diffs, and supports extensions through custom instructions, agent skills, and MCP servers. The stated result is faster development and guided debugging while workflows remain aligned with Unity Catalog governance, performance, and data quality standards; future plans include background failure response and cluster right-sizing.


### [Securely send first-party conversion signals with Snapchat Conversions API on Databricks Marketplace](https://yomu.fyi/post/securely-send-first-party-conversion-signals-with-snapchat-conversions.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Nikhil Gaekwad, Jen McNamee, Katy Yuan
- Published: Apr 28, 2026

Snapchat Conversions API is now available as a Databricks Marketplace listing that lets teams activate first-party conversion signals directly from governed Lakehouse data. The included pre-built notebook reads gold-layer tables, batches web, app, and offline events, and supports deduplication with Snap Pixel, allowing server-side signals without double-counting. Because it runs in the Databricks environment, Unity Catalog preserves governance, lineage, security, and access controls while teams control what data is shared and when. The post says fuller server-side coverage can raise Event Match Quality, while citing that advertisers who improved their EQS from “Poor” to “Good” saw 26% higher ROAS and 49% lower CPI. It also describes fresher custom audiences and Lookalikes plus more complete deduplicated measurement.


### [How leading tech companies are killing the builder's tax with Lakebase](https://yomu.fyi/post/how-leading-tech-companies-are-killing-the-builder-s-tax-with-lakebase.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Amey Banarse, Madelyn Mullen
- Published: Apr 27, 2026

The post argues that AI-native applications face a “builder’s tax” because operational databases and lakehouse analytics are separated by CDC, ETL/ELT, and reverse ETL pipelines. This architecture can leave AI systems with stale data while duplicating governance and shifting engineering effort toward orchestration and failure management. It presents Lakebase, a fully managed serverless Postgres engine integrated with the Databricks Data + AI Platform, as a shared operational foundation where applications, agents, analytics, and governance use the same data. The proposed pattern combines a lakehouse intelligence layer, a low-latency Lakebase execution layer, and a continuous learning loop, with pgvector supporting agent memory and semantic search. Reported customer examples include faster feature onboarding, reduced on-call disruption, higher agent throughput, and consolidated data estates, though the results vary by organization.


### [Powering KPMG UK Audit's AI future with Databricks](https://yomu.fyi/post/powering-kpmg-uk-audit-s-ai-future-with-databricks.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Mark Wallington, Greta Nasai
- Published: Apr 27, 2026

KPMG UK is evolving its Audit data platform with Databricks to support AI-enabled analytics while preserving governance, rigour, and professional standards. The program converges structured data, advanced analytics, and AI on a unified cloud-native Lakehouse, with Databricks SQL replacing core SQL Server workloads and Delta underpinning the platform. Lakebridge assessed migration complexity, while Databricks-hosted large language models including Claude Sonnet and Genie Code helped convert T-SQL, refactor stored procedures, modularize queries, and suggest Delta-based optimizations under engineer review. This reduced refactoring time by around 60% and enabled modernization of more than 400 scripts and stored procedures in roughly three months. Databricks SQL Serverless provides elastic compute for spiky workloads, while Genie offers traceable, version-controlled SQL and Delta Sharing supports governed data exchange.


[Newer posts](https://yomu.fyi/company/databricks/page/18.md) · [Older posts](https://yomu.fyi/company/databricks/page/20.md)