Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,318 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
What is Artificial Intelligence (AI)?
Artificial intelligence (AI) is a branch of computer science that enables machines to perform tasks associated with human intelligence, including learning, reasoning, pattern recognition and decision-making. Modern AI generally learns patterns from large datasets, tunes internal weights and parameters during training, evaluates outputs on held-out data, and applies the resulting model during inference to classify, predict, generate content or trigger actions. Most organizations fine-tune existing foundation models rather than train from scratch, while output quality remains dependent on the completeness, bias and quality of training data. The page separates reactive machines and limited memory from theoretical theory of mind and self-aware systems, and distinguishes today’s narrow AI from theoretical general AI and superintelligence. It also describes generative AI, common applications, risks including hallucinations, bias, privacy and security gaps, and governance, concluding that practical adoption depends on real problems, trusted data and responsible oversight.
Data Engineering for AI: A Practical Guide for Data Professionals
Data engineering for AI extends traditional ETL by adding requirements for model-ready data, unstructured inputs, feature engineering, governance, and production monitoring. The guide addresses data professionals building or scaling AI infrastructure, covering ingestion, architecture, data quality, privacy compliance, generative AI, and career development. It emphasizes shared feature schemas, pipeline data contracts, lineage, statistical validation, drift detection, and PII stripping or anonymization as responsibilities shared across data engineering and data science. For generative AI systems, it describes preparing RAG pipelines by ingesting and chunking documents, creating vector embeddings, and indexing them for semantic retrieval, while evaluating vector databases for latency, scale, and integration. It concludes that reliable AI depends on fresh, accurate, compliant data and ongoing operational and architectural review.
Data Warehouse Types: A Complete Guide to Architectures and Use Cases
A data warehouse is a centralized repository for structured data, supporting complex queries, reporting, and business intelligence rather than transaction processing. The guide compares architectures by scale, latency, cost, scope, ownership, and governance. Enterprise data warehouses integrate organization-wide sources through ETL, apply cleansing and validation, and provide a governed source of truth, while data marts focus on departmental analysis and may be dependent or independent. Operational Data Stores replicate current or recent operational data for reporting refreshed from minutes to hours, whereas virtual, cloud, hybrid, and lakehouse designs trade physical consolidation, scalability, flexibility, and governance differently. The comparison also frames lakehouses as combining open-format data lake flexibility with warehouse-style governance and transactional reliability.
What is Row-Level Security?
Row-level security (RLS) is a database access control that filters which table rows a user may read or change according to identity, role, or session context. It supports multi-tenant SaaS, regional segregation, departmental access, compliance, healthcare, and partner use cases, while typically layering table-level grants with column-level security or data masking. The database applies a policy or predicate automatically at query time across BI dashboards, notebooks, ad-hoc SQL, APIs, and third-party tools, with separate USING and WITH CHECK rules for reads and writes. On Databricks, Unity Catalog row filters use SQL user-defined functions and apply across Databricks SQL, notebooks, jobs, and connected BI tools. The source recommends simple predicates, indexed keying columns, identity-based testing, and explicit enforcement for both reads and writes.
Beyond LoRA: Can you beat the most popular fine-tuning technique?
Parameter-efficient fine-tuning (PEFT) reduces the memory required to adapt open models, enabling fine-tuning with a fraction of the memory and, in some cases, quantized models. LoRA is dominant because it freezes base weights and trains a small set of added parameters, but its popularity may also reflect ecosystem momentum and uneven comparisons. The post presents shared-condition benchmarks for language-model math reasoning and image generation, using identical models, datasets, code, hardware, and measures including test performance, VRAM, forgetting or drift, runtime, and checkpoint size. The results show that LoRA performs well but is not consistently best: other techniques can outperform it on one or more axes, especially in the image-generation benchmark. The recommended choice is therefore to compare techniques against a use case’s needs and constraints, including quantization support, layer coverage, adapter merging, and downstream serving compatibility.
Payment Fraud Detection: How Banks and Businesses Stop Fraudulent Transactions
Payment fraud detection is presented as a real-time practice for identifying and blocking unauthorized transactions amid evolving card-not-present, account takeover, authorised push payment, friendly, and card-testing attacks. Modern systems combine predefined rules, machine learning, behavioral analytics, device fingerprints, geolocation, transaction history, and velocity signals to calculate risk scores within milliseconds, declining or routing high-risk payments to review. Rules are fast and interpretable but require continual tuning, while machine-learning and behavioral models can identify complex or user-specific patterns that static thresholds miss. The source also emphasizes prevention controls such as MFA, tokenization, 3D Secure, and device binding, alongside unified real-time data infrastructure that supports cross-channel signals, continuous retraining, and deployment into payment authorization flows.
What is an AI Agent Harness?
An AI agent harness is the software infrastructure around a large language model (LLM) that lets it act on tasks rather than only answer prompts. The model reasons and chooses actions, while the harness connects tools, execution environments, memory, external systems and safety controls. Many agents follow a reason–act–observe loop: the model proposes an action, the harness executes it, captures the result and returns that result as context for the next decision. Production harnesses typically include system prompts, tool execution, sandboxes, filesystems and durable storage, context management, feedback and self-verification, guardrails, human-in-the-loop approvals, and observability; their design increasingly shapes reliability as model capabilities converge.
Data Warehouse Modernization: Roadmap, Architecture, and Services
Data warehouse modernization addresses legacy systems that cannot scale efficiently with growing data volumes, real-time analytics, machine learning, and self-service access. The proposed roadmap spans two to four years for large estates, moving from assessment and architecture design through high-impact workload migration, governance embedding, and optimization rather than relying on a risky big-bang cutover. Its target architecture favors a lakehouse or enhanced cloud data warehouse, with open formats such as Apache Iceberg or Delta Lake, separate compute and storage, and Bronze, Silver, and Gold layers supporting incremental ELT and lineage. The source says modernization can reduce infrastructure maintenance costs by 30–50%, compress query latency from hours to seconds, and halve redundant ETL pipelines, while also improving governance for sensitive data and enabling BI, machine learning, and generative AI workloads on a shared foundation.
The open ecosystem around ClickHouse
ClickHouse’s open ecosystem spans clients, dashboards, ingestion pipelines, projects built on its engine, and agents that query it, with 2,600+ contributors, 240,000+ commits, and hundreds of millions of downloads cited. It describes open standards including OpenTelemetry for observability, Model Context Protocol (MCP) for AI tooling, and Iceberg and Delta Lake for lakehouse data, plus Iceberg REST catalogs, Apache Polaris, and Unity Catalog. The surrounding stack includes ClickStack, chDB, Langfuse, LibreChat, official language clients, a Kubernetes operator, Postgres bridges such as PeerDB and pg_clickhouse, and integrations with Airflow, dbt, Grafana, Fivetran, and Sigma. Specific updates include full Iceberg v2 support, Iceberg writes in 25.8, Paimon in 25.10, chDB v4’s DataStore API, and a Python client v1.0 with native async and 4x faster cold starts. The post frames this openness as supporting flexible tooling and community-led growth, reporting more than 200 integrations while noting that the ecosystem’s next direction remains unknown.
A2UI + MCP Apps: Combining the best of declarative and custom agentic UIs
Agentic UIs must balance deep customization with seamless integration: MCP Apps provide standard-web flexibility in an iframe, while A2UI renders declarative JSON through the host application's native components. The post presents three architectural patterns for combining them, including A2UI over MCP, where MCP servers return application/a2ui+json resources for native rendering and can separate static MCP Resources from dynamic Tool calls. This arrangement avoids iframe-related visual fragmentation, keeps backend tool access separate from frontend rendering, and supports rendering across React, Flutter, or Angular through a predefined component catalog. The broader combination also reserves MCP Apps for highly tailored experiences and describes A2UI in MCP Apps, where an embedded app can process A2UI responses and route user actions through an AppBridge; the conclusion recommends choosing among patterns according to project constraints.
Clustering billions of products for agentic commerce with Catalog API
Shopify’s Catalog must reconcile billions of listings whose merchants use different product and variant structures, so AI shopping agents can identify the same real-world product. Product clustering groups related listings under a Universal Product Identifier, with precision prioritized over recall because wrong products can lead to incorrect purchases while missed variants remain findable. The intra-store system starts with rules-based singleton detection, then uses ANN and sparse average linkage for pre-chunking, and applies a two-stage LLM process that proposes clusters and critiques them using the core value proposition framework. Dynamic per-chunk strict JSON schemas enforce complete product coverage, support arbitrary chunking, and improved recall by 8% on the Toloka evaluation dataset in one schema change. The resulting pipeline is presented as a scalable intra-store foundation, while cross-store clustering into global UPIs is identified as the next step.
Agentic Resource Discovery: Let agents search
Agentic Resource Discovery (ARD) is a draft, open specification for discovering agent capabilities at runtime, addressing a model in which users and developers must pre-install, integrate, and maintain each MCP tool, skill, or A2A agent. It defines a static ai-catalog.json manifest and a dynamic POST /search registry API, using indexed signals such as publisher identity, representative queries, compliance attestations, and tags to move capability selection outside the LLM. Hugging Face's Discover Tool adapts Hub semantic search into ARD catalog entries for running Spaces, exposing skills, MCP servers, or raw Space metadata according to the requested media type and wrapping agents.md files into installable SKILL.md artifacts. The tool is available through the hf CLI, REST, and MCP, while planned work includes federation modes and Hub support for profile-level manifests.
What is data pipeline architecture?
Data pipeline architecture is the end-to-end blueprint for collecting, processing, storing and delivering data from source systems to people, applications and models. It distinguishes logical design, which defines stages and responsibilities, from physical design, which assigns tools and infrastructure to those stages. The common four layers are ingestion, processing and transformation, storage, and serving and consumption, with orchestration and observability spanning the pipeline. The brief compares batch, streaming, Lambda, Kappa and medallion patterns, explaining trade-offs involving freshness, cost, complexity and operational burden. It also contrasts ETL with ELT and presents governance, monitoring and right-sized processing as reliability practices, concluding that architecture should match the use case and balance freshness, cost and reliability.
Cart Assistant: Agentic Grocery Shopping on Uber Eats
Cart Assistant addresses grocery requests that begin as lists, recipe images, or vague plans and turns them into draft carts shoppers can review and edit. It uses a multi-prompt state graph that assigns planning, candidate retrieval, semantic relevance, constraint enforcement, quantity selection, guardrails, assembly, and content refinement to focused stages. LLMs handle ambiguity and structured outputs, while backend APIs, deterministic arithmetic, validation, optimization, and aggregation manage inventory, pricing, eligibility, sellable units, and cart construction. The design separates retrieval language from reasoning context, preserves quantity provenance, supports parallel item processing, and uses deterministic checks plus an LLM judge against synthetic and production-derived cases. According to the conclusion, this decomposition makes the system more controllable and inspectable while producing useful draft carts without removing shopper review, editing, and decision-making.
What is customer segmentation?
Customer segmentation divides an existing customer base into smaller groups based on shared demographic, geographic, psychographic, behavioral, firmographic or value-based characteristics. Unlike market segmentation, it uses first-party data about existing customers, and a customer can belong to multiple segments at once. The guide distinguishes rule-based, survey-based, RFM, k-means, decision-tree and AI/ML-driven methods, with choices depending on data maturity and business goals. Effective implementation defines an objective, audits and unifies sources, resolves duplicate identities, selects a method, validates segments and measures outcomes. It also describes Databricks' CustomerLake capabilities, including governed Customer 360, agentic identity resolution, natural-language segmentation through Genie and bidirectional activation connectors; stated benefits include improved retention, conversion, customer lifetime value and marketing efficiency.
What is an open lakehouse? Open data standards, explained.
The piece defines an open lakehouse as a lakehouse whose storage, table format, processing engine, catalog, and ML and AI tooling use open standards and remain interchangeable. It contrasts this architecture with warehouses, lakes, and proprietary lakehouses, emphasizing low-cost object storage, ACID transactions, governance, schema guarantees, and the ability to change engines without rewriting data. Its reference stack combines open table formats such as Delta Lake and Apache Iceberg with Apache Parquet, Apache Spark, Unity Catalog, and MLflow, while allowing engines including DuckDB, Trino, and PyIceberg to work on the same data. The article also distinguishes open standards from open-source code, explains that a table format is only one layer of the stack, and states that the components can be self-hosted or consumed through a managed service.
13 Social Media Holidays to Celebrate This July
This calendar presents 13 unofficial social media holidays for July 2026, giving brands occasions to engage audiences through patriotism, culture, kindness, humor, and food. It lists dates from International Joke Day on July 1 through International Day of Friendship on July 30, including two observances on July 18 and two joke-focused days. Each entry pairs the observance with a suggested engagement prompt, such as asking for favorite jokes, running polls, inviting emoji-only stories, collecting acts of kindness, or encouraging friends to share appreciation. The recommendations span educational, entertaining, inspirational, and community-oriented content, with examples tied to Independence Day, World Emoji Day, and National Ice Cream Day. It is positioned as a way to shape social content during the second half of 2026.
How the University of Nebraska-Lincoln teams up with Uber to power late-night rides
At the University of Nebraska-Lincoln, undergraduate and graduate students needed reliable late-night transportation during the academic year. ASUN began powering rides with Uber in 2025, and the Husker Late Night Ride program launched in 2026 with safety as its primary goal. The program provides five $10 UberX vouchers per semester for rides between 9pm and 3am, seven days a week, with trips limited to a three-mile radius of UNL’s City and East campuses or Lincoln Airport. Students register online, receive vouchers at official UNL email addresses, and add them to Uber accounts for automatic application in eligible zones and times. Funded by Nebraska Athletics alcohol-sale revenue, the program has attracted over 6,000 students and delivered 15,236 Uber trips, with Friday accounting for 37.8% of rides; ASUN uses Uber for Business Dashboard to track spending, impact, parameters, and reports, while pay-per-performance supports cost efficiency.
What a difference 10 years of open source makes
ClickHouse marks ten years since becoming open source under the Apache 2.0 license, crediting more than 2,600 contributors and a decade of monthly releases for its growth into one of the world’s most widely deployed analytical databases. Recent engineering work spans joins, updates and deletes, JSON, full-text search, vector search, query acceleration, integrations, and developer tooling. Join-focused changes made the TPC-H SF100 workload 26× faster between versions 22.4 and 26.4, while statistics-based reordering reduced one six-table query from 3,903.7 seconds to 2.7 seconds. The project also publishes open benchmarks, including ClickBench, JSONBench, CostBench, and PostgresBench, with scripts, configurations, data, and results intended for independent validation. Its next phase continues to emphasize community contributions and broader ways to use ClickHouse.
Skip the learning curve: rethinking data migration for real outcomes
Data migrations are presented as high-risk, costly initiatives whose technical completion can delay adoption and strategic value, especially during infrequent warehouse transitions. The proposed alternative combines migration, modernization, and value creation in parallel, using experienced specialized partners and AI-enabled automation for code conversion, data-quality validation, and pipeline modernization. Rather than lifting and shifting legacy workloads, teams are urged to simplify architectures, retire unnecessary components, reduce technical debt, and align data with business needs while validating progress continuously. Progressive decommissioning reduces the “double-bubble” period in which old and new systems run together, helping costs fall as workloads move instead of waiting for final completion. The Migrate & Modernize Program connects organizations with partners, and the post reports faster cutovers, reduced migration costs, and complex workloads entering production ahead of schedule among early participants.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.