Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,318 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
Beyond the spreadsheet: How Databricks is delivering the modern CFO in financial services
Financial-services CFOs are being asked to act as Strategists and Catalysts, but fragmented legacy systems, T+1 batch processing, opaque reporting logic, and mismatched business semantics keep them focused on stewardship and retrospective operations. The proposed answer is Databricks as a unified, governed platform combining real-time streaming, centralized lineage, data and AI, with Unity Catalog, Lakeflow, Genie, and Agent Bricks addressing trust, latency, access, and model reproducibility. Unity Catalog can connect semantic definitions and end-to-end lineage from transactions through regulatory reports and models, while Lakeflow supports continuous ledger and liquidity processing. The post cites a global bank reducing liquidity-reporting processing from 10 hours to 8 minutes and Nationwide Insurance reporting a 5-point combined-ratio improvement and 3-point expense-ratio improvement. It presents a Lakehouse-based CFO stack as a shift from reporting historical results toward real-time capital management and previews AI-driven deposit and PPNR modeling.
IoT in Manufacturing: Strategy, Components, Use Cases, and Challenges
IoT in manufacturing is presented as an operational architecture for collecting machine data, improving production and supply-chain visibility, and preventing equipment failures under strict latency, reliability, and safety requirements. The guide covers device selection, sensor placement, tiered sampling, edge-to-cloud pipelines, platform criteria, security, compliance, workforce training, and a phased deployment roadmap. Its technical split sends high-frequency vibration and acoustic signals to edge gateways for local processing while cloud systems support stateful analytics, predictive maintenance scoring, and model training; protocols such as MQTT, OPC-UA, AMQP, and Modbus are evaluation criteria. Reported examples include multi-factory OEE monitoring across more than 200 production lines, reducing reporting lag from 24 hours to under five minutes, digital twins for maintenance simulation, and IoT logistics routing. The action plan recommends starting with predictive maintenance and OEE monitoring on one line before scaling.
Personalization Strategies for Media Companies
Media companies are expected to tailor content and messaging across channels, with personalization programs typically targeting lower churn, deeper sessions, and higher subscription revenue. The proposed foundation starts with an audit of CRM, email, warehouse, and customer data platform sources, followed by required-field definitions, ownership mapping, consent tagging, and ingestion-quality checks. Teams then map segment behaviors and conversion or retention paths, including negative signals, before selecting a CDP with real-time ingestion, identity resolution, unified profiles, segmentation, activation, and governance. Identity rules should connect anonymous and authenticated activity across web, mobile, and connected TV, while event-level streaming delivers messages through email, push, in-app, and advertising channels with minimal latency. The roadmap recommends an MVP comprising unified ingestion, a homepage recommendation surface, and triggered re-engagement email, with performance, churn, revenue lift, and model drift monitored through iteration.
A Modern AI Risk Management Framework
AI systems introduce probabilistic behavior, model drift, adversarial manipulation, bias, and auditability challenges that traditional IT risk practices may not address. The framework described combines NIST AI RMF, EU AI Act, and ISO/IEC 23894:2023, using Govern, Map, Measure, and Manage as its operational structure while recognizing regulatory and societal context. It recommends cross-functional governance, clear risk ownership, an AI-BOM covering systems, data flows, dependencies, and accountability, plus continuous measurement of fairness, explainability, data quality, security vulnerabilities, and harm likelihood and severity. Across the AI lifecycle, the text identifies risks including data poisoning, model drift, malicious library injection, prompt injection, hallucinations, and platform vulnerabilities, and points to the AI RMF Playbook for checklists, timelines, and governance artifacts.
Flow generation through natural language: An agentic modeling approach
Shopify built a tool-calling agent for Sidekick that converts natural-language requests into Shopify Flow workflows, addressing the cold-start problem by working backward from thousands of anonymized production workflows. The team generated synthetic queries and tool trajectories, fine-tuned Qwen3-32B, and changed Flow’s nested JSON DSL into an equivalent Python representation connected by a round-trip transpiler. Using Python improved syntactic correctness by 22 points and semantic correctness by 13 points on a 300-example benchmark, while matching production tool names, ordering, responses, and prompts proved essential to avoid degradation. Production evaluation exposed a 35% gap masked by benchmark parity, so Shopify built a weekly feedback loop that scores conversations, filters training data, identifies slice gaps, retrains, and deploys. The resulting agent serves most production traffic, runs 2.2 times faster at 68% lower cost, and outperforms closed models.
Building an Enterprise Data Management Strategy
Enterprise data management (EDM) is presented as an organizational capability for governing, integrating, securing, and operationalizing data across its lifecycle as volumes and regulatory scrutiny increase. The strategy addresses silos, inconsistent schemas, undocumented transformations, compliance exposure, and the difficulty of supporting analytics and AI across fragmented systems. It combines governance councils and ownership, automated profiling and quality thresholds, classification with RBAC and encryption, reusable ETL/ELT pipelines, lineage tracking, and architecture choices such as lakehouses and medallion patterns. Unity Catalog is described as a unified governance layer for reads and writes across open formats including Delta Lake and Apache Iceberg, while federation can provide access to external catalogs without copying data. Execution begins with cross-functional sponsorship, a high-impact pilot, measurable KPIs, and quarterly reviews; the source frames EDM as continuous rather than one-time work.
Real-Time Decisioning for AI Agents: Why you Need a Customer Context Layer First
Real-time decisioning for AI agents depends on more than customer records: it requires a customer context layer that captures what people are doing now and connects that behavior to identity. The post positions the data platform as the center of a composable martech architecture, with agents and applications operating on shared data in open formats such as Apache Iceberg and Delta Lake. Snowplow’s described approach places structured event collection, schema validation, enrichment, and identity resolution before data reaches the platform, while operating in the customer’s cloud environment. It distinguishes historical profiles from granular behavioral event streams and argues that agent interaction outcomes should return to the foundation as first-class events. The resulting four-stage loop—collect, resolve and enrich, serve, and learn—supports simultaneous real-time and historical context, with decision quality dependent on source data coherence.
Keeping the Cursor app stable
Cursor's desktop app faces disruptive crashes as usage and features such as subagents, instant grep, and browser use have expanded, with most failures caused by out-of-memory conditions. The team measures instability across Electron's multi-process architecture using telemetry, per-session and per-request crash rates, minidumps, stack traces, crash breadcrumbs, and dashboards that update within minutes. Its two-pronged debugging strategy combines feature-level experiments and oversize-message instrumentation with bottom-up analysis based on a crash watcher, Chrome DevTools Protocol (CDP) stacks, automated daily investigations, opt-in heap snapshots, and low-rate allocation profiling. Findings led to chunking large data, killswitches, lifecycle fixes for leaks, and process isolation for extensions, while prevention uses Bugbot rules, stress tests, garbage collection, performance tests, and automated rollbacks. OOM-per-session fell 80% from its late-February peak and OOM-per-request fell 73% since March 1.
Concurrency Control in DBMS: How Locking, MVCC and Optimistic Strategies Keep Data Consistent
The guide explains how database management systems coordinate simultaneous transactions to preserve consistency and serializability, preventing dirty reads, lost updates, non-repeatable reads and phantom reads. It compares lock-based control, including shared and exclusive locks and two-phase locking, with MVCC, which gives transactions consistent snapshots and lets readers and writers proceed without blocking. The discussion covers pessimistic and optimistic strategies, isolation levels, deadlocks, retries, lock-granularity trade-offs and MVCC’s storage and garbage-collection costs, including PostgreSQL’s VACUUM. For lakehouse workloads, Delta Lake uses a read-validate-commit process with optimistic concurrency, snapshot isolation and conflict detection for concurrent writes. The source presents Databricks Lakebase as a managed implementation offering optimistic concurrency control, snapshot isolation for reads and write-serializable isolation for writes without lock or deadlock management.
LLM apps: Real-world use cases, core building blocks and risks to know
LLM applications are presented as production systems that extend beyond chatbots by connecting foundation models to external data, tools, APIs, memory and workflows. Modern categories include assistants and copilots, retrieval-augmented generation (RAG), autonomous agents, multi-agent systems and local or on-device inference, each with distinct capabilities and trade-offs. A typical RAG pipeline uses vector search over a curated knowledge base, then supplies retrieved passages to the model; chunking, source freshness and retrieval accuracy affect results. Production systems also require orchestration, evaluation, observability, governance, guardrails, audit trails and human-in-the-loop checkpoints, especially when agents can call tools or execute workflows. The source distinguishes prompt engineering, which guides behavior without changing model weights, from fine-tuning, which adapts models to domain-specific tasks, and describes a unified path combining data, models, retrieval, orchestration, evaluation and governance for production deployment.
AI and the Future of Cybersecurity: Why Openness Matters
Mythos is presented as a frontier AI system that rapidly finds and patches software vulnerabilities, with the post arguing that its surrounding system matters more than the language model alone. Its recipe combines substantial compute, software-relevant training data, vulnerability-probing and patching scaffolding, speed, and some autonomy; together, these elements can uncover vulnerabilities, find exploits, and build patches. The post argues that open code and tooling can distribute detection, verification, coordination, and patch propagation across communities, while reducing capability asymmetry between attackers and defenders. For defensive use, it recommends semi-autonomous agents with prespecified actions, human approval, auditable logs, and private deployment using open components and existing security tools. This approach is especially relevant to high-stakes organizations that need inspectable, controllable systems and protection for sensitive data.
From Static Policies to Autonomous Insurance: How AI enables Real-Time Coverage
Insurance products often rely on annual pricing, periodic risk assessment, and coverage changes tied to major life events, even though driving behavior, digital fraud exposure, and travel risk can change in seconds. The post presents embedded insurance and usage-based insurance as a shift toward contextual, just-in-time coverage delivered through apps, checkout flows, vehicle purchases, and partner platforms. Its automotive example combines telematics, geospatial data, and real-time weather signals to update risk scores and premiums from trip behavior such as braking, following distance, mileage, location, and environmental conditions. A Databricks Lakehouse architecture registers devices, ingests telemetry through Zerobus, curates data in a governed Medallion architecture, and exposes features through Lakebase and Databricks One, with Unity Catalog supporting access controls, lineage, explainability, and audit trails. The proposed trajectory leads from embedded coverage toward autonomous insurance whose policies continuously adapt to risk.
What Is a Transactional Database?
Transactional databases handle large volumes of short, real-time read and write operations for systems such as orders, payments, account balances, and customer records. They support online transaction processing (OLTP) by treating related steps as one logical unit: successful work commits, while failures roll back the full transaction. ACID—atomicity, consistency, isolation, and durability—protects integrity under concurrency and failures, with row-oriented storage optimized for individual-record access, locking and isolation levels coordinating concurrent work, and write-ahead logging (WAL) supporting crash recovery. The source positions these databases as foundations for banking, e-commerce, healthcare, reservations, and telecommunications, while noting that row-oriented execution is less suitable for large analytical workloads and that horizontal scaling is more complex than in distributed NoSQL systems.
What is pgvector?
pgvector is an open-source PostgreSQL extension that adds vector storage, indexing, and similarity search for embeddings alongside relational application data. It supports semantic search, recommendations, retrieval-augmented generation (RAG), image similarity, anomaly detection, and deduplication without requiring a separate vector database for many workloads. The extension uses the vector data type and distance metrics including L2, cosine similarity, and inner product, with HNSW and IVFFlat indexes offering different speed and memory trade-offs. Vector queries can also combine relational filters with Postgres full-text search for hybrid retrieval. pgvector is positioned as a practical choice for existing Postgres deployments, while pgvectorscale can extend its usefulness as data and query demands grow; dedicated vector databases become more relevant at larger scales, and Databricks AI Search serves complementary lakehouse workloads.
Banks don't have an AI problem — they have a data platform problem
The post argues that banks’ difficulty scaling AI stems less from model capability than from fragmented data, weak governance, and limited real-time access. Examples from CBA Live 2026 include model drift in credit scoring, fragmented customer signals, and a collections model that predicted, with 85% accuracy, how many days newly delinquent accounts would take to cure using governed data. It also presents Erica’s 3.2 billion interactions since 2018 as evidence that production AI requires continuous tuning, monitoring, and edge-case management, while generic models decay in frontline settings. The proposed Databricks architecture combines Lakehouse, Unity Catalog, Lakeflow, Lakebase, MLflow, Model Monitoring, Online Feature Store, Genie, and Agent Bricks to support governed analytics, low-latency decisions, auditable models, and constrained agent actions. Its conclusion is that a shared data platform should precede additional AI use cases because it improves deployment speed, trust, explainability, and regulatory defensibility.
Open Platform, Unified Pipelines: Why dbt on Databricks is Accelerating
dbt provides structure for transforming raw data into curated datasets, but the post argues that its effectiveness depends on the underlying platform. Databricks is presented as an open lakehouse combining storage, compute, governance, orchestration, and monitoring to reduce duplicated data, fragmented permissions, and operational overhead. Lakeflow Jobs runs dbt alongside ingestion, downstream actions, retries, and dependencies, while Unity Catalog supplies persistent permissions, documentation, lineage, and fine-grained controls. The integration also supports Delta Lake and Apache Iceberg, query tags for cost tracking, Photon, Predictive Optimization, Liquid Clustering, and dbt materialized views. The post concludes these capabilities provide openness, integrated operations, governance, and price-performance for dbt workflows, citing one customer’s consolidation of notebooks and 7+ source systems and more than 2,900 customers already running dbt on Databricks.
Building a Unified Pipeline for AI Token Spend
AI token spend is volatile, consumption-based, and distributed across teams, making usage and cost difficult to monitor. Ramp's pipeline ingests LiteLLM and OpenRouter events through authenticated, tenant-scoped webhooks, Kafka, and ClickHouse, then aggregates them for REST dashboards and AI-generated forecasts. LiteLLM uses callbacks with token counts and metadata, while OpenRouter sends OTLP traces; ReplacingMergeTree deduplicates replayed events using business_id, source, and event_id. Customers can connect gateways with an API key, configure LiteLLM callbacks or OpenRouter Broadcast, and pass attribution metadata. The resulting visibility supports breakdowns by provider, model, department, user, project, and use case, and exposed phantom Gemini reasoning tokens after a LiteLLM upgrade, geo pricing premiums, runaway loops, and prompt bloat.
What is Generative AI in Marketing?
Generative AI in marketing creates content, insights and recommendations that can personalize experiences, optimize campaigns and improve performance. Unlike traditional analytics, which mainly reports on past results, it produces net-new outputs such as ad copy, audience segments, product recommendations, visual assets and strategic summaries. A typical workflow prepares campaign, customer and brand data, grounds or fine-tunes models, generates outputs, applies targeting and optimization, and uses human review and refinement. It distinguishes pretrained tools such as ChatGPT, Claude and Perplexity from customized models and broader AI transformation, noting trade-offs between speed, investment, relevance and strategic alignment. Successful adoption depends on high-quality, consented data, governance, privacy controls, human oversight, monitoring and cross-functional ownership; the conclusion presents AI as a force multiplier rather than a replacement for human creativity and judgment.
Ecom-RLVE: Adaptive Verifiable Environments for E-Commerce Conversational Agents
Ecom-RLVE extends the RLVE framework from single-turn reasoning puzzles to multi-turn, tool-augmented e-commerce conversations. EcomRLVE-GYM supplies eight environments spanning product discovery, substitution, cart building, returns, order tracking, policy QA, bundle planning, and multi-intent journeys, with procedural problem generation, a 12-axis difficulty curriculum, and algorithmically verifiable rewards. Each episode requires tool use and world-state changes; rewards combine task completion, efficiency, and a penalty for recommending product IDs not retrieved, while malformed outputs and illegal calls fail immediately. Adaptive scheduling advances each environment independently when its success rate is reliable. An initial study trained Qwen 3 8B with DAPO for 300 steps on cart building and reported progressively increasing difficulty reached, consistent with a steady learning signal.
How we built Stripe Credits: A programmable, auditable way to pay your Stripe fees
Stripe Fee Credits replaces ad-hoc discounts and manual cash adjustments with a programmable, auditable virtual balance dedicated to eligible Stripe fees. The architecture has three layers: a standalone gRPC credits engine for metadata, policies, lifecycle, and audit history; logical financial accounts built on Money Movement and Storage; and fee-settlement orchestration that applies eligible credits before the primary balance. Logical accounts carry ledger balances without representing real cash, while platform checks restrict Originated Money Transfers to logical-to-logical movements and exclude these accounts from cash sweeps. Prepaid and Stripe-funded credits share the user experience but use separate funds flows: prepayments create deferred revenue and collect tax at drawdown, whereas Stripe-funded credits are booked as Stripe costs with taxes written off. The resulting infrastructure supports incentives, prepaid annual fees, and other programs across 26+ countries, with Dashboard balances and real-time auditability, while avoiding bespoke engineering for each commercial model.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.