Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Sciene AI Companion: building an autonomous Customer Success platform on Databricks
Sciene built AI Companion for Quartile’s Customer Success organization, where CSMs support more than 1,000 brands and previously spent substantial time preparing decks, reconstructing context, and investigating account changes. The platform addresses personalization at scale, high-volume content generation, and root-cause diagnosis by combining account data, CSM communication styles, company principles, and cross-domain business data. Its Email Hub cuts reply time from 15–30 minutes to about three minutes, Meeting Hub reduces preparation for 80+ slide decks from over two hours to around 10 minutes, and Account Flagging reduces diagnosis of flagged accounts from 30+ minutes to about five. Databricks provides the shared governed foundation: Delta Sharing supplies data without copies, Lakebase stores operational state, and SQL Warehouses serve analytical, AI, and operational workloads from the same tables. The design keeps CSMs responsible for judgment while giving them current context for customer interactions.
What’s new with Unity Catalog at Data + AI Summit 2026
At Data + AI Summit 2026, Unity Catalog announcements position the catalog as a runtime governance layer for enterprise data and AI, organized around control, context, and choice. Control additions include Unity AI Gateway for governing models, agents, MCP services, skills, and tools; contextual service policies can allow, deny, or require approval for runtime actions, while budgets, hard caps, tracing, and guardrails address spend, investigation, and safety. Context additions include Glossary and Domains for business meaning and scoped asset organization, plus Metrics that standardize KPIs for SQL, BI tools, APIs, and agents; Genie Ontology is described as a continuously learned enterprise context layer. Choice additions span cross-cloud and cross-region addressability, managed disaster recovery, Delta and Iceberg interoperability, multimodal and geospatial types, and open sharing of data, AI assets, and applications across organizations.
Agent Bricks: Data + AI Summit 2026
At Data + AI Summit 2026, Databricks announced Agent Bricks as a comprehensive developer platform for building and operating agents, extending a product launched the previous year. The announcement frames the core agent loop as only 1% of the work, with token capacity, deployment, security, evaluation, monitoring, context, and sharing forming the remaining infrastructure burden. Agent Bricks addresses choice, context, and control through support for multiple proprietary, open-source, and custom models, any agent harness, MCP-connected data, Genie Ontology, managed memory, document intelligence, sandboxes, and governed tools. Unity AI Gateway adds catalogs, fine-grained access controls, budgets, traffic routing, contextual policies, monitoring, and registry support for agents, tools, and models. Databricks says more than 100,000 agents have been built and customers including AstraZeneca, 7-Eleven, Fox Corporation, and Block have shipped agents on the platform.
AI governance at Data + AI Summit 2026: What’s new with Unity AI Gateway
Databricks announces new Unity AI Gateway capabilities for governing enterprise AI as organizations operate multi-model, multi-agent, and multi-vendor estates connected to models, MCP services, APIs, and tools. The update adds unified spend visibility, granular attribution, hard spend caps, and smart routing, alongside Unity Catalog support for registering and governing models, MCP services, agents, and skills. Contextual Service Policies, in Beta, can allow, deny, or require approval for actions based on users, agents, models, tools, services, or request and response contents, with guardrails for risks such as PII exposure and prompt injection. The announcement also covers end-to-end tracing, coding-agent analysis with Genie, incident investigation with Lakewatch, ecosystem integrations, and Managed Omnigent on Databricks in Beta.
Lakeflow: A new era of agentic data engineering
Databricks announces a major evolution of Lakeflow, its unified platform for data engineering across ingestion, transformation, and orchestration, with capabilities centrally governed by Unity Catalog. Genie Code and generally available Lakeflow Designer support agentic and no-code pipeline development, while Genie ZeroOps monitors production assets, analyzes failures, proposes fixes, and validates them in a governed sandbox before human approval. Lakeflow Connect expands to more than 100 managed connectors, and Zerobus Ingest adds Kafka-compatible, gRPC, REST, SDK, and OpenTelemetry interfaces for high-volume event ingestion. Real-Time Mode for Spark Declarative Pipelines reaches Public Preview with end-to-end latency as low as 5 milliseconds, alongside declarative APIs and expanded Lakeflow Jobs integrations. The release also adds data-readiness triggers and external orchestration for systems including Snowflake, REST APIs, Slack, and PagerDuty.
Introducing Genie ZeroOps: Put your data and AI operations on autopilot
Genie ZeroOps is an autonomous background agent for monitoring and operating data and AI assets, including jobs, pipelines, tables, and ML models. It continuously detects visible and silent failures, uses Unity Catalog lineage and platform observability to assess root causes, generates remediation through development workflows, and verifies fixes in isolated sandboxes. These environments use shallow, zero-copy table clones, scoped permissions, and network isolation, so proposed changes run against real data without touching production or applying anything before approval. For ML workloads, the agent can diagnose degraded predictions, train a candidate on corrected features, evaluate it against the production model’s existing eval suite and criteria, and support live-traffic ramping when it is measurably better. Genie ZeroOps is entering private preview in the coming weeks, initially supporting jobs, pipelines, tables, and ML workloads; Apps and Lakebase databases are on the roadmap.
Announcing Lakebase Search: agent-native retrieval built into Lakebase Postgres
Lakebase Search is a beta offering on AWS and Azure that adds hybrid vector and full-text retrieval to Lakebase Postgres. It uses the lakebase_vector and lakebase_text extensions to keep retrieval, memory, operational data, and hybrid search in one backend. lakebase_vector retains pgvector types and operators, applies RaBitQ clustering and compression for 32x smaller indexes, and targets more than 1B vectors, while lakebase_text replaces GIN with object-storage-optimized BM25 ranking. A tiered cache keeps hot data on NVMe and places colder data in object storage; the source reports lower memory needs, faster index builds, and cold-cache startup than standard pgvector HNSW in its LAION-100M benchmark. The extensions also combine vector similarity and keyword relevance with reciprocal rank fusion in a single SQL query, enabling joins and tenant filtering alongside transactional workflows.
Announcing the new Databricks Startup Program
The Databricks Startup Program has been updated for venture-backed, early-stage startups building with data and AI. Qualifying companies can receive up to $200,000 in credits across Databricks and Neon, along with hands-on technical guidance, partner and community access, and connections to go-to-market teams and founder communities. The program is aimed especially at startups that recently raised institutional funding from pre-seed through Series A, and it is intended to provide an app backend, data, and AI stack from idea through product-market fit. Databricks and Neon are presented as providing a database and access to foundation models on day one, while Databricks supplies analytics, data warehousing, AI systems, and enterprise-grade governance as companies launch. Applications are available through the Databricks Startup Program.
Unifying Data and Governance in the Agentic Era: What’s New with Azure Databricks
At Data + AI Summit 2026, Azure Databricks announced capabilities aimed at moving enterprises from experimental AI pilots to production-grade automated workflows by unifying data, productivity tools, marketing, and governance on Azure. Its Agentic Data foundation introduces LTAP, combining analytical data, streaming pipelines, and live application transactions in one lakehouse storage copy; Lakebase adds a managed serverless Postgres engine with copy-on-write branching, while Lakehouse//RT targets millisecond responses for high-concurrency workloads. Genie integrations for Microsoft Teams, M365 Copilot, Excel, and SharePoint bring governed lakehouse intelligence and ingestion into daily work, alongside tools for agents, applications, pipelines, and autonomous operations. CustomerLake adds Profile Agents and Campaign Agents for customer profiles and personalization, while Genie Ontology and Unity AI Gateway provide semantic context, rate limits, content filtering, and spend controls.
Introducing Genie One, Genie Agents, and Genie Ontology
Databricks announces Genie One, Genie Agents, and Genie Ontology to help enterprises answer business questions and act on data whose context is scattered across dashboards, queries, documents, tickets, and chats. Genie One connects data and business tools through Lakehouse federation, Lakeflow Connect, native integrations, Slack, Teams, mobile apps, schedules, alerts, document creation, custom skills, and MCP support. Genie Agents evolve Genie Spaces into domain-specific agents that can reason over structured and unstructured data, execute multi-step workflows, and be created from a prompt. Genie Ontology builds a permission-aware living graph from enterprise assets, weighting sources by authority, usage, certification, and freshness. In an internal 28-question benchmark, Genie answered 84.5% correctly on the first attempt and delivered twice the speed of the strongest coding agent.
Accelerate search queries with full-text search indexes on Databricks
Databricks introduces full-text search indexes in Beta on Databricks Runtime 18.2 to accelerate substring and keyword queries on large open-format tables without changing their layouts. The indexes tokenize text columns into a compact lookup structure mapping tokens to matching rows; at query time, the engine uses it to identify candidate files and skip most of the table. They are maintained asynchronously, require no query hints, preserve complete results when stale by scanning indexed and non-indexed data as needed, and support Unity Catalog managed Delta and Iceberg tables on serverless and classic compute. A Trust and Safety team reported a substring search running more than 100x faster on a petabyte-scale table, while Liquid clustering remains complementary because it optimizes column-value filters rather than text within fields.
Introducing CustomerLake: The Agentic CDP embedded in Databricks
Databricks announces CustomerLake, an Agentic Customer Data Platform embedded natively in its lakehouse, bringing Customer 360, identity resolution, audience building, campaign automation, activation, and personalization alongside governed data and AI models. The announcement addresses fragmented identities, stale audiences, manual campaign workflows, and the duplication and governance burden created by separate martech systems. CustomerLake uses Unity Catalog and Lakehouse Federation to access customer data across Databricks, Snowflake, Google BigQuery, cloud object storage, operational databases, and other enterprise systems, while Profile Agents create business-ready profiles and Campaign Agents build audiences, recommend actions, activate channels, and optimize engagement. Its operating model is described as embedded, democratized, and autonomous, with Agentic Identity Resolution combining deterministic, probabilistic, and agentic workflows. CustomerLake is now available in Private Preview and launches with an open partner ecosystem.
What is customer segmentation?
Customer segmentation divides an existing customer base into smaller groups based on shared demographic, geographic, psychographic, behavioral, firmographic or value-based characteristics. Unlike market segmentation, it uses first-party data about existing customers, and a customer can belong to multiple segments at once. The guide distinguishes rule-based, survey-based, RFM, k-means, decision-tree and AI/ML-driven methods, with choices depending on data maturity and business goals. Effective implementation defines an objective, audits and unifies sources, resolves duplicate identities, selects a method, validates segments and measures outcomes. It also describes Databricks' CustomerLake capabilities, including governed Customer 360, agentic identity resolution, natural-language segmentation through Genie and bidirectional activation connectors; stated benefits include improved retention, conversion, customer lifetime value and marketing efficiency.
What is an open lakehouse? Open data standards, explained.
The piece defines an open lakehouse as a lakehouse whose storage, table format, processing engine, catalog, and ML and AI tooling use open standards and remain interchangeable. It contrasts this architecture with warehouses, lakes, and proprietary lakehouses, emphasizing low-cost object storage, ACID transactions, governance, schema guarantees, and the ability to change engines without rewriting data. Its reference stack combines open table formats such as Delta Lake and Apache Iceberg with Apache Parquet, Apache Spark, Unity Catalog, and MLflow, while allowing engines including DuckDB, Trino, and PyIceberg to work on the same data. The article also distinguishes open standards from open-source code, explains that a table format is only one layer of the stack, and states that the components can be self-hosted or consumed through a managed service.
Introducing Lakehouse//RT: Real-Time Performance on a Unified Lakehouse
Databricks introduces Lakehouse//RT, a real-time data warehouse designed for operational analytics, BI, app serving, and observability workloads. It is powered by Reyden and is intended to deliver millisecond performance directly on lakehouse data without copying it into a separate serving layer. Preview participants saw up to 16x better performance, with response times as low as 10ms on smaller datasets, sub-100ms on larger ones, and sub-100ms latency at 12,000 queries per second on standard analytical benchmarks. Tests covering concurrency, dataset scale, and complex TPCDS queries report low latency where alternatives slowed or failed. Lakehouse//RT is in Beta for select read-only workloads, with incremental autoscaling and automatic baseline compute sizing.
13 Social Media Holidays to Celebrate This July
This calendar presents 13 unofficial social media holidays for July 2026, giving brands occasions to engage audiences through patriotism, culture, kindness, humor, and food. It lists dates from International Joke Day on July 1 through International Day of Friendship on July 30, including two observances on July 18 and two joke-focused days. Each entry pairs the observance with a suggested engagement prompt, such as asking for favorite jokes, running polls, inviting emoji-only stories, collecting acts of kindness, or encouraging friends to share appreciation. The recommendations span educational, entertaining, inspirational, and community-oriented content, with examples tied to Independence Day, World Emoji Day, and National Ice Cream Day. It is positioned as a way to shape social content during the second half of 2026.
Case study: TriMet unlocks same-day paratransit with an innovative partnership
TriMet’s 2024 partnership with Uber, TransDev, and Trapeze added same-day capability to its LIFT paratransit service in Metro Portland. Before LIFT+, trip management spanned multiple systems, while cancellations and manual coordination increased operational burden and weakened the rider experience. In April 2024, riders began booking through LIFT’s web platform or call center, after which TriMet’s integrated Trapeze PASS TripBroker used the Uber Plug-in to select dedicated vehicles or Uber based on rider profiles, requests, trip efficiency, and fleet capacity; it could also reschedule cancellations and delays. Since launch, TriMet has delivered more than 47,300 Uber trips and reduced average cost per trip from $78 to $54, while riders gained same-day flexibility and trip details including estimated arrival times.
How the University of Nebraska-Lincoln teams up with Uber to power late-night rides
At the University of Nebraska-Lincoln, undergraduate and graduate students needed reliable late-night transportation during the academic year. ASUN began powering rides with Uber in 2025, and the Husker Late Night Ride program launched in 2026 with safety as its primary goal. The program provides five $10 UberX vouchers per semester for rides between 9pm and 3am, seven days a week, with trips limited to a three-mile radius of UNL’s City and East campuses or Lincoln Airport. Students register online, receive vouchers at official UNL email addresses, and add them to Uber accounts for automatic application in eligible zones and times. Funded by Nebraska Athletics alcohol-sale revenue, the program has attracted over 6,000 students and delivered 15,236 Uber trips, with Friday accounting for 37.8% of rides; ASUN uses Uber for Business Dashboard to track spending, impact, parameters, and reports, while pay-per-performance supports cost efficiency.
Building Production-Scale Visual AI for Claims and Returns at Swiggy Instamart
Swiggy Instamart built two vision-language-model systems to handle more than 40,000 daily claims involving wrong, expired, poor-quality, damaged, and packaging-related items, while reducing reliance on human verification. The Image Validation Model (IVM) compares customer uploads with catalog images, routes each category through a specialised LangGraph StateGraph handler, and combines its structured verdict with fraud, confidence, and business-rule signals. Reverse Quality Check (RQC) compares delivery executives’ pickup photos with the original claim images, allowing matching returns to proceed and prompting retries when they do not. The architecture keeps visual assessment probabilistic but makes routing, aggregation, escalation, and financial decisions deterministic, with traces for category, outputs, confidence, cost, and latency. After more than six months in production, the systems are associated with a 6% customer-satisfaction increase, 20,000 fewer weekly Image Validation tickets, 98.5% RQC accuracy, and over 5,000 daily RQC tickets resolved without human intervention.
Uber Advertising Launches New Solutions to Connect Brands On & Off Uber
Uber Advertising has launched new solutions that connect brands with consumers across Uber rides, Uber Eats, and external channels. Offers on Uber include Destination Offers, Homescreen Ride Offers, Ride Offers on Journey, and planned Sponsored Upgrades, combining trip context with incentives and premium ride upgrades. On Uber Eats, Brand Takeover, Item Showcase, and Offers Spotlight provide prominent placements, shoppable item features, and targeted offer visibility during discovery and ordering. Offsite Ads extends campaigns to Meta and Google Shopping, using observed Uber behaviors to target audiences and connect external engagement with orders and store visits; the program is being piloted with selected advertisers. Reported findings include 81% openness to offers while riding, a 45% higher click-through rate for one Miller Lite campaign, and double-digit item attachment after Item Showcase views, with results varying by campaign.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.