Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,318 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
What is Document AI?
Document AI uses machine learning, natural language processing (NLP) and optical character recognition (OCR) to extract, classify and understand information from structured, semi-structured and unstructured documents. Unlike OCR alone, it interprets layout and context, turning files such as invoices, contracts and emails into structured, actionable data through ingestion, OCR, layout parsing, entity extraction, classification, validation and, when needed, human review. Modern systems add large language models for summarization, document Q&A and zero-shot extraction, but hallucination risk makes validation and human oversight essential, particularly in regulated settings. The guide also describes Databricks Document Intelligence, which processes and stores documents alongside organizational data under Unity Catalog, using AI Functions, Variant and Lakeflow Jobs to create governed, queryable workflows without moving data between systems.
Ten years of ClickHouse in open source
ClickHouse’s tenth-anniversary history describes its path from an internal web-analytics experiment to an open-source analytical database with more than 2,000 contributors. The original problem was processing rapidly growing, real-time logs while answering flexible reports quickly; tested MySQL extensions and standalone analytical databases did not meet the stated scale of 100 billion records per day and 500 columns. The project built column-oriented storage, lightweight compression, custom aggregation structures, background merges, and query support for grouping, aggregation, filtering, and sorting. Released publicly on June 15, 2016, ClickHouse is presented as a level-three open-source project with public contribution guidelines, code review, testing, CI, documentation, and a roadmap, while contributors receive credit even when implementations are substantially rewritten.
How API changes flow into Stripe's developer products
Stripe describes how API changes are propagated across its developer-product suite, including SDKs in seven languages and three release channels, the CLI, Shell, API Explorer, API Reference, changelog, and internal tools. Historically, v1 workflows used OpenAPI or a Ruby-based DSL while v2 APIs used protobuf-based DSLs and an internal API specification, forcing teams to maintain parallel generators and leaving some products without v2 support. Stripe chose OpenAPI as a shared representation and built a pipeline whose generated artifacts are tailored by variant, release phase, and API version, with vendor extensions carrying Stripe-specific metadata. Automated validation, OpenAPI diffs, pre-release comparisons, versioned snapshots, and downstream publishing now connect API definition changes to documentation, code samples, SDK and CLI releases, while older v1 versions remain difficult to regenerate with newly added metadata.
From Wall Street to Data Platforms
Kim Hatton, Databricks’ Global Financial Services Marketing Leader, describes how two decades in regulated financial-services marketing led her toward technology and data-centered strategy. She says financial institutions still need to unlock value while navigating compliance, but their divisions and systems make a unified customer view difficult. Her account points to Unity Catalog’s unified governance and single source of truth for breaking down silos and supporting requirements including GDPR, customer identity, and sensitive workloads. It also describes Lakebase’s separation of compute and storage for faster ML/AI agent experimentation, alongside Genie’s plain-language data analysis, which can reduce work that otherwise takes months and expensive third parties. Hatton connects these tools with faster, more confident marketing and accurate decision-making in regulated workflows, while also describing Databricks’ inclusive, high-energy culture.
What is enterprise intelligence?
Enterprise intelligence (EI) is presented as an organization-wide capability combining business intelligence, knowledge management, enterprise search and AI to turn structured and unstructured data into decisions and actions. Unlike traditional BI, which centers on dashboards, reports and structured data, EI connects these capabilities through a shared architecture and governed business context. The described stack includes a lakehouse-based data foundation, batch and streaming pipelines, governance, semantics, analytics, search, machine learning, generative AI and a decision layer. Shared definitions such as “active customer” and “monthly revenue” are intended to keep dashboards, queries and AI agents aligned, while maintained context addresses knowledge that becomes stale as the business changes. The result described is a common trusted source from which people, applications and agents can produce insights and initiate actions.
Enabling Evolutionary Database Development: Database branching with Lakebase, the conclusion
The post concludes a series on how copy-on-write database branching in Databricks Lakebase changes team-scale evolutionary database development without changing its underlying methodology. For a team of fifty developers, long-running tier branches and ephemeral feature branches form a parent-linked promotion hierarchy, replacing separately provisioned environment instances and enabling promotion by merge, rollback by repoint, and computable schema divergence. Governance is declared once and inherited per branch, with policies intended to prevent transitions that contradict the parent chain; Unity Catalog captures metadata for attribution and audit. The DBA's role becomes platform engineering, while agents operate inside an executable SCM state machine with documented inputs, outputs, schema validation, and enforced gates. An optional TDD layer adds dedicated roles, acceptance-criterion scenarios, RED-GREEN-REFACTOR cycles, and artifact contracts, and the conclusion presents the resulting workflow as operational for human and agent practitioners.
Results from first Anthropic Public Record
Anthropic’s first Public Record survey measures how Americans viewed AI in November and December 2025, using a nationally representative online sample of 51,993 people aged 16 and over. Respondents most often hoped AI would help cure diseases (48%), while job loss was the leading concern (64%), followed by cognitive dependency (56%) and misinformation (52%). More than 70% supported government involvement in AI, especially around privacy, child safety, and liability, but only 15% trusted AI companies to decide how AI should be developed and used. The results describe broad agreement across partisan, geographic, and educational groups, with differences usually reflecting intensity rather than direction; Anthropic says the survey will be repeated to track changing attitudes as capabilities and adoption develop.
Unlocking semantics for AI: How Mercedes-Benz Korea built trusted “Talk to Data” at scale
Mercedes-Benz Korea piloted a “Talk to Data” architecture that extends its Databricks analytics foundation with a governed semantic layer for enterprise AI, rather than treating the effort as a chatbot project. The design moves Power BI DAX KPI logic into Unity Catalog Business Semantics and Metric Views, keeping sources, joins, measures, dimensions, comments, and synonyms alongside governed Lakehouse data. Genie spaces use curated metric views for domain questions, while Agent Bricks composes persona-based agents, with Unity Catalog enforcing row- and column-level access. An automated DAX-to-Metric-View transpiler parses semantic models, maps tables, generates draft definitions, flags non-automatable measures, and reports conversion gaps. The documented playbook combines gold-layer curation, KPI validation, regression testing, Genie optimization, persona agents, and Databricks Apps; the pilot reports AI answers aligned with established KPI definitions and BI reporting logic.
Production-Ready Agents Need A Production-Ready Data Platform
AI development teams face constant shifts in model providers and agent frameworks, demanding data platforms that provide scalable, real-time context management. Agentic workloads require blending unstructured enterprise data, short-term session state, and persistent long-term memory. MongoDB addresses these requirements through its native JSON document model and integrated retrieval capabilities, combining full-text search, vector search, and hybrid search directly over operational data. Customer implementations such as DevRev, ElevenLabs, and Adobe rely on Atlas to achieve sub-100 millisecond hybrid retrieval and handle billions of requests. Additionally, MongoDB is collaborating with LangChain and ecosystem partners to establish open reference architectures and shared interfaces for portable agent memory across frameworks.
Simplifying Data and Product Integrations with a Data Abstraction Layer
Uber built a domain-agnostic Data Abstraction Layer (DAL) to decouple data consumers from changing physical datasets and simplify advertiser reporting. The DAL is an RPC service that accepts a logical table, schema, time range, and filters; logical tables define the interface, while configuration describes locations, relationships, availability windows, and rollup transforms. It generates database-specific queries, runs them in parallel across systems such as an internal OLAP database, Docstore, and Apache Hive, and assembles responses with aggregation and rollup behavior determined by the request. Removing a dimension can produce an account-level response with metrics automatically rolled up, while mixed realtime and daily tables can cover different portions of a requested time range. For advertiser reporting, the approach reduced new report delivery from several weeks or months to under two days and enabled dynamic filters, report builders, and third-party API integrations.
Making FlashAttention-4 faster for inference
FlashAttention-4, originally designed for pre-training, required modifications to efficiently handle LLM inference workloads dominated by the memory bandwidth-limited decode phase. Modal’s contributions include adjusting parallelism strategies from query parallelism to key/value parallelism, implementing split KV to parallelize across KV tiles, and adding a heuristic for optimal splits. They added support for FP8 inputs, arbitrary KV page sizes via cp.async loads, and reduced wasted work for small query sizes by using a single query tile. The kernel also gained extended GQA packing for irregular head ratios, using cp.async for query tiles to bypass TMA alignment restrictions. These changes, measured in throughput improvements up to 3x for single-token decode and up to 2.4x for small page sizes, demonstrate the leverage of flexible parallelism and tile-based programming for inference optimization.
Stop building data products. Start building data services.
Howden’s rapid acquisition pace exposed limits in an enterprise data model built around one product per use case, downstream quality checks, and dashboard-driven consumption. Group Chief Data Officer Barry Panayi describes shifting to open, governed data services, moving mastering and quality checks closer to ingestion, and codifying reconciliation in the Accord data model. On Databricks, the company consolidated more than 100 sources of record, standardized pipelines and shared code, and built reusable assets for cross-domain analytics, while continuing to productionize models as consistent services. The account argues that AI agents require a composable services layer, and that insight lag—the time between data existing and being usable—matters more than freshness; conversational analytics through Genie also reduced dashboard-building work.
Scaling AI Through Data Fluency
Aer Lingus is redirecting a significant share of its IT and change spending from traditional maintenance toward a Databricks-powered data foundation, addressing legacy systems that trap information in departmental silos. Dave O’Donovan says the airline spent the past 18 months prioritizing platform development, governance, data quality and data literacy rather than chasing each new AI product. Databricks was selected for a unified lakehouse architecture, with data warehousing, Genie’s plain-English querying and real-time operational data intended to broaden access beyond specialist teams. At Aer Lingus’s Operations Control Center, combining sensor and operational inputs gives teams a fuller real-time view for disruption decisions, while commercial teams use live insights to adjust pricing. The transformation also includes a Data Literacy Academy, a 75/25 capacity split between foundational work and innovation, a 20-person Continuous Improvement team, and experiments with agents for business-case development and CFO review.
How Rivian drives trusted, AI-powered decisions at the speed of thought with Databricks
Rivian is building electric vehicles and services that require fast, trusted decisions across manufacturing, supply chain, finance, service and operational planning, while business users need reliable metrics and insights. Using Databricks AI/BI, Genie, Unity Catalog metric views, Databricks Apps and AI-assisted engineering, the company is consolidating dashboards, semantic definitions, permissions, sensitive data and AI-powered workflows on one governed foundation. Rivian migrated a massive multi-domain dashboard base in less than six months, is standardizing more than 50 metrics, and worked with Databricks as a design partner on roughly 58 product features. The resulting self-service analytics and operational applications cut supply-chain monitoring time by 60 to 70%, reduce inventory investigations from over 30 minutes to under two, predict stock-out risk more than four days ahead, and reduce some ingestion setup time by more than 60%, supporting AI-powered decisions without competing versions of the truth.
Quick: An internal hosting platform for the AI era
Shopify built Quick to make internally created prototypes, dashboards, tools, and games easier to share than deploy, giving employees a secure URL for a folder of HTML and assets. Launched internally in July 2025, it places sites behind Identity-Aware Proxy, serves bucket content through NGINX and gcsfuse, and deploys with a wrapper around gcloud rsync. A service in front of CloudSQL provides database and realtime APIs, while server-held credentials support calls to Shopify’s AI proxy; file uploads, BigQuery access, WebSockets, and employee identity are also available. Quick now hosts more than 50,000 sites, has been used by over half of Shopify employees, and runs on one $200-per-month VM, with constraints, rate limiting, and a migration from Node to Go supporting its operation.
Under the Hood: Scaling Responsible AI at Uber
Uber describes a company-wide Responsible AI program designed to govern models developed across its many platforms and teams as AI becomes central to products and services. Its five pillars are visibility, explainability, governance, education, and adoption, implemented through a centralized Model Catalog, automated metadata, feature attribution, lifecycle controls, and employee learning resources. Model Cards provide searchable model descriptions, metrics, deployment details, and documentation, while Michelangelo integrates global, local, and deep-learning explanations using PFI, SHAP-based methods, TreeSHAP, and integrated gradients. Governance is shifted left into product and engineering reviews, with completion prompts before deployment, and adoption work brings existing models into scope through lifecycle-aware, technology-agnostic, context-sensitive controls. The program gives engineers and oversight teams shared visibility into model behavior and risk while aiming to support continued AI growth without losing accountability and trust.
How QRT powers real-time research and risk management at petabyte scale
Qube Research & Technologies (QRT), a global quantitative investment manager headquartered in London, uses ClickHouse Cloud for a data platform that serves researchers and for near-real-time risk monitoring, management, and P&L. Founded in 2018 as a cloud-native firm after the spinout of Credit Suisse’s proprietary trading arm, QRT chose a cloud-based approach rather than inheriting decades of on-premise infrastructure. The platform supports two major systems, while ClickStack consolidates logs, metrics, and traces for observability; alerts, log queries, and metric aggregations run against a single ClickHouse instance using standard SQL. QRT says its previous database was constrained by writer and reader counts and total cluster load, whereas ClickHouse provides greater speed and scalability, and that a unified monitoring backend simplifies maintenance while remaining reliably fast.
NYC Bike Courier Regulations FAQ
New York City couriers who deliver by bike or e-bike must meet requirements covering platform identification, protective equipment, bicycle hardware, and Department of Transportation safety training. The FAQ explains how to submit a New York or New Jersey mailing address in the Uber Driver app, receive an Uber-branded vest and photo ID, request replacements, and troubleshoot shipping or address changes. It lists required equipment including a helmet, front lamp, bell, side reflectors, and working brakes, while noting that replacements are available six months after the first order and brake reimbursements require dated, itemized receipts for eligible purchases on or after April 8, 2026. Training requires a DOT account using the Uber-associated email, a quiz, and certificate sharing through a 10-digit Worker ID or the app.
How an Agent Built a 3D Paris Gallery by Chaining Two Hugging Face Spaces
A coding agent built a 3D Paris gallery by chaining two Hugging Face Spaces: an image-generation Space produced dark-background monument images, and VAST-AI/TripoSplat reconstructed each single image as a 3D Gaussian splat. It then handled integration work by correcting the splats’ Y-down orientation, auto-framing monuments, compressing .ply files to .ksplat at about one-third the size, and building and deploying a static Three.js viewer. The Spaces’ agents.md files provided schemas, call and polling templates, file-upload instructions, and authentication guidance, allowing the agent to call them without a client library. The same pipeline produced six-monument galleries for Egypt and Japan after the prompts changed. The account presents callable, documented multimedia components as a way to reduce integration work and make new multimedia applications cheaper to create.
Managing Elasticsearch Reindex at Scale: Performance, Reliability, and Observability
Palantir’s document store needs to repair or rebuild Elasticsearch indices when mappings, settings, cluster versions, or index state change, while continuing to serve live search and writes. Its reindex machinery reconstructs schema-aware indices from the strongly consistent database, rather than relying on Elasticsearch’s _reindex API, and is designed to limit manual work and resource consumption across many deployments and schemas. For an online reindex, it creates a physical shadow index with the intended mappings and settings, backfills it from the database, dual-writes current API changes to both indices, and promotes the shadow index after completion; searches remain on the primary during the rebuild. Cross-cluster migrations use a separate machine with cluster-specific metadata, configuration, index and alias evolution, and preservation of old-cluster indices to support rollback.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.