Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,318 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
What is a Relational Data Model?
The relational data model organizes structured data in tables of rows and columns, with SQL used to access and manage it. Introduced by Edgar F. Codd in 1970, it uses schemas, domains, keys, constraints, and relationships to define and connect data. A relation schema specifies a table's structure, while a relation instance is the data held at a particular moment; primary, candidate, super, foreign, and composite keys support identification and referential integrity, and one-to-one, one-to-many, and many-to-many relationships model associations between tables. Functional dependencies inform normalization, whose 1NF, 2NF, and 3NF stages reduce redundancy and insertion, update, and deletion anomalies, though heavier normalization can require more joins. The model is used in banking, e-commerce, healthcare, education, and enterprise systems, where constraints and ACID transactions help preserve accuracy and reliability.
What is Intelligent Document Processing?
Intelligent document processing (IDP) uses AI to extract, classify, validate, and structure information from PDFs, images, emails, forms, and other structured, semi-structured, or unstructured documents. Its pipeline combines document ingestion and preprocessing with OCR, document classification, NLP, layout analysis, data validation, human review for uncertain results, and delivery into downstream systems such as ERP, CRM, or accounting software. The post explains how embeddings support classification, how continuous learning adapts models to new layouts and corrections, and how metrics such as accuracy, throughput, confidence scores, and model drift support optimization. IDP can reduce manual effort, improve accuracy, lower costs, scale document-heavy workflows, and supply structured data for analytics, automation, AI agents, and retrieval-augmented generation, while high document variability and edge cases still make human-in-the-loop review relevant.
What are Analytic Applications?
Analytic applications are packaged business intelligence systems designed to measure, analyze and improve performance for defined business domains. They combine data integration, modeling, visualization, reporting and workflow support so technical and non-technical users can turn operational data into decisions. Typically, they connect ERP, CRM and supply chain systems with a data warehouse or lakehouse, process historical and real-time data, and deliver dashboards, reports, scorecards or embedded insights through preconfigured schemas, connectors, governance controls and business logic. Unlike general-purpose BI tools built for open-ended exploration, they support standardized, domain-specific workflows such as sales performance management, financial planning, risk assessment and inventory optimization. Organizations can start with descriptive analytics and adopt predictive or prescriptive capabilities as data maturity increases, while choosing standalone or embedded deployment based on workflow needs and data readiness.
What is a Cloud
A cloud-based database management system stores, manages, and queries data on third-party cloud infrastructure instead of local or on-premises servers. It retains capabilities such as query optimization, security, transactions, and support for relational SQL, NoSQL, data warehouse, and in-memory workloads, while operating across distributed nodes. Organizations can self-manage database software on cloud virtual machines or use DBaaS, where the provider manages the software and infrastructure; connections typically use SQL, APIs, or web consoles. Virtualized resources can scale, and providers may handle patching, backups, maintenance, and failover, while replication across availability zones supports recovery and availability. The comparison with on-premises systems covers ownership, scalability, cost models, maintenance, accessibility, control, compliance, total cost, and emerging capabilities including AI integration, vector search, autonomous operation, multicloud, hybrid, and serverless architectures.
What is Agentic Analytics?
Agentic analytics is an approach in which autonomous AI agents continuously monitor data, detect patterns and anomalies, generate contextual insights, and take actions with minimal human intervention. Unlike traditional business intelligence, which relies on people to query dashboards and interpret reports, these systems operate as goal-directed decision loops that can recommend or trigger alerts, forecasts, workflows, and system changes within defined boundaries. Their architecture combines data access, agent orchestration, LLM-based reasoning, a semantic layer, action integrations, and feedback signals. The source identifies faster insight, lower reporting overhead, reduced analytics backlogs, and potentially better forecasting as benefits, while stressing governance, auditability, security, monitoring, and human oversight to control errors and unintended outcomes.
Memory scaling for AI agents
Memory scaling is proposed as an agent’s performance improving when it accumulates past conversations, feedback, interaction trajectories, and business context in persistent external memory. Unlike model-parameter or inference-time scaling, it relies on selective retrieval to supply relevant information without filling long context windows. In MemAlign experiments on Databricks Genie Spaces, adding labeled examples raised test scores from near zero to 70%, about 5% above an expert-curated baseline, while average reasoning steps fell from about 20 to 5; filtered user logs produced similarly strong gains, exceeding 50% after 62 records. Production use requires storage and retrieval for episodic and semantic memories with organizational and user-level scoping, plus controls that preserve permissions and sensitivity labels.
Dynamically Scaled Activation Steering
Activation steering has emerged as a way to guide generative models toward desired behaviors, but uniform interventions can reduce performance when steering is unnecessary. DSAS, or Dynamically Scaled Activation Steering, addresses this by separating when to steer from how to steer and adapting intervention strength across layers and inputs. At generation time, it computes context-dependent scaling factors that modify existing steering transformations, and it can also be jointly optimized end-to-end with the steering function. Combined with existing methods, DSAS improves the reported Pareto front between toxicity mitigation and utility preservation, while adding minimal computational overhead; experiments also apply it to a text-to-image diffusion model for modulating specific concepts and show token-level interpretability through the strength and location of steering.
coSTAR: How we ship AI agents at Databricks fast, without breaking things
Databricks describes coSTAR, a methodology for testing and iteratively improving long-running, stochastic AI agents without relying on manual review for every change. It addresses nondeterministic outputs, slow executions, cascading failures, and subjective quality by treating scenarios as fixtures, MLflow traces as a reusable harness, and judges as an automated test suite. Its two coupled STAR loops align judges with human experts using a Golden Set, then use those judges to refine the agent until it passes the scenarios. Agentic judges selectively explore traces with tools instead of receiving entire, potentially thousands-line executions, while persisted traces allow rescoring without rerunning agents. The approach can also reuse judges for production monitoring, but remains limited by manually authored scenarios, judge-calibration costs, coverage gaps, overfitting, and difficult root-cause attribution.
Business Analytics Tools: A Complete Guide for Data
Business analytics tools have evolved beyond static reporting as organizations seek explanations, forecasts, and immediate actions. This guide categorizes visualization and dashboard platforms, self-service analytics, statistical tools, spreadsheets, and SQL-based query tools, then examines how AI adds natural-language querying, anomaly detection, summaries, and forecasting. It argues that inconsistent, duplicated, or differently governed data can undermine even sophisticated dashboards. Lakehouse architecture combines data-lake scalability with warehouse governance, performance, and SQL compatibility, enabling tools such as Tableau, Power BI, and Looker to use broader, fresher datasets alongside analytics and AI workloads. Reported examples include 94% faster BI queries at Anker Innovations, 98% faster query performance at AnyClip, and consolidation across 120+ global analysts at JLL.
MLOps Frameworks: A Complete Guide to Tools and Platforms for Production ML
The guide explains how MLOps frameworks close the gap between notebook experimentation and reliable, scalable machine-learning production by applying automation, version control, and continuous delivery across the lifecycle. It describes experiment tracking, model registries, workflow orchestration, feature stores, deployment and serving, and monitoring as the capabilities needed to make training reproducible, prevent training-serving skew, automate pipelines, and detect degradation after release. MLflow is presented as a modular open-source option with Tracking, Model Registry, Models, and Projects, supporting run history, lifecycle stages, standardized packaging, and reproducible execution across environments. Kubeflow suits Kubernetes-standardized organizations and large distributed GPU workloads, while Metaflow, cloud-native services, and unified platforms such as Databricks are positioned as alternatives according to developer experience, cloud alignment, or operational preferences.
What is Machine Learning? How It Works, Types and Use Cases
Machine learning (ML) is a subset of artificial intelligence that learns statistical patterns from data to make predictions or decisions without hard-coded rules. The guide distinguishes ML from generative AI, which creates new content, and from deep learning, a specialized ML approach using multilayered neural networks that learns features automatically and typically requires more data and computing power. It describes a workflow of preparing and splitting data, training models by comparing predictions with actual outcomes, and adjusting parameters to reduce error, illustrated with image classification and customer-churn scoring. It also covers supervised, unsupervised, and reinforcement learning, applications including computer vision and natural language processing, newer architectures such as transformers and Mamba, and MLOps practices for deployment, monitoring, and retraining.
How agentic software development will change databases
AI agents are changing software development from a slow, linear process into rapid evolutionary experimentation, creating many small, ephemeral applications and databases. The post identifies three resulting database requirements: near-zero marginal cost with elastic growth, native branching for rapid variants, and open ecosystems that agents can use reliably. Lakebase addresses branching through O(1) metadata copy-on-write at the storage layer, so agents can branch production or test state without physical data copying and pay only for experiment compute. Its decoupled, serverless compute can scale in sub-second intervals and down to zero, while standard open Postgres page formats in cloud object storage keep data accessible to external engines and tools. Lakebase telemetry reports about 10 branches per project on average, some nested beyond 500 iterations, and roughly four times as many databases created by agents as by humans.
How Addepar scales investment workflows with Databricks AI agents
Addepar, a global technology and data platform, manages nearly $9 trillion in assets and moved from older systems to a unified data intelligence platform on Databricks running on AWS. It ingests hundreds of disparate data feeds, standardizes and enriches them, and delivers results through products, APIs and data sharing. Since adoption, pipeline costs fell 60% versus legacy infrastructure, saving more than $2 million, while new pipelines and integrations became five times faster. Addison, a native AI experience embedded in the platform, combines governed portfolio Q&A, proactive insights and action-oriented workflows, using live data and Addepar’s core calculations engine. Addepar is adopting Databricks Agent Bricks, including Supervisor Agent flows with Genie-powered analytics, while Foundation Model Serving and managed MLflow support governed model serving, traceability, evaluation and iteration for human-supervised agentic workflows.
Why agentic analytics starts with a well
Agentic analytics increases the need for trustworthy semantics and governance because AI systems must interpret business definitions, trace outputs, and support follow-up questions beyond static dashboards. Nick Eayrs argues that fragmented metrics across BI tools, proprietary semantic layers, and duplicated pipelines erode trust, slow decisions, and increase engineering cost. The proposed foundation is an open, interoperable semantic layer where certified metrics such as revenue, churn, and customer lifetime value are explicitly defined, reusable, and accessible through SQL to BI, AI interfaces, notebooks, and agents. Organizations should establish governed data foundations and a catalog first, certify metrics next, then add AI and evaluate its outputs. Examples include NTT Docomo reducing manual log-analysis time from 66 hours to 6 hours and Net One Systems cutting support response time by 75 percent.
How Zalando built a unified data foundation for AI and analytics on Databricks
Zalando describes how it is building a unified data foundation for analytics and AI across a microservices-driven ecosystem serving more than 50 million active customers and 7,000 brands and partners. Its architecture combines private domain-owned catalogs for autonomous development with a centrally governed shared catalog that exposes cross-team data through Dynamic Views, enabling identity-based policies, GDPR checks, column classification, and auditability. Databricks Metric Views provide a governed semantic layer where business logic and KPI definitions are centralized and reusable across SQL, dashboards, notebooks, and AI agents. Genie uses those metrics plus enriched metadata to answer natural-language questions more reliably, although poorly curated or unaggregated data can increase costs; Zalando plans broader enterprise adoption through curated spaces and additional orchestration.
Powering clinical research innovation: How TriNetX uses Databricks to accelerate drug development
TriNetX operates a federated real-world health data network spanning more than 230 healthcare organizations and needed infrastructure to make growing volumes of RWD easier to use for clinical research. Databricks now provides the centralized lakehouse architecture, consolidating electronic health record data while supporting custom datasets, consulting, ML models, proprietary algorithms, APIs and self-service access. TriNetX is extending this foundation with a beta Query Assistant for natural-language analysis and a Support Assistant prototype built with Databricks’ Agent Bricks, requiring no programming expertise for the former. In 2025, pharmaceutical clients reduced protocol amendments by up to 50%, while an AI-enhanced site-identification effort achieved a 63% site acceptance rate and nine-day average response time. Other models suggest enrollment conversion could rise from 33% to 85%, with expanded APIs and Genie exploration planned for 2026.
What is AIOps?
AIOps applies AI and machine learning to IT operations to analyze logs, metrics, traces, events, alerts, and related system data. It detects anomalies, correlates events, uses topology and dependencies to identify likely root causes, and supports guided or automated responses. The guide positions AIOps between observability and action, reducing noise and adding context without replacing observability, DevOps, or human judgment. Its workflow includes data ingestion, normalization, anomaly detection, event correlation, root-cause analysis, automation and orchestration, and collaboration workflows, with human-in-the-loop review recommended for higher-risk production changes. It contrasts domain-centric and domain-agnostic approaches and recommends evaluating coverage, depth, integrations, setup effort, and organizational fit before expanding automation.
Building cloud-native PACS on AWS
Multi-hospital networks face rising imaging volumes, siloed on-premises PACS, radiologist routing bottlenecks, and storage that must expand for 7 to 10 years of retention. The proposed pattern uses a hub-and-spoke hybrid architecture: local PACS instances preserve LAN-speed reading, while a centralized AWS archive provides cross-facility discovery, disaster recovery, and lifecycle-managed storage. Modalities send DICOM through C-STORE to local VNA servers, which normalize and compress objects; asynchronous replication sends images to Amazon S3, while Amazon Aurora PostgreSQL stores metadata and routing information. The AWS deployment spans two Availability Zones, places PACS services behind Network Load Balancers, and includes CloudFront and AWS WAF. Storage guidance distinguishes S3 Intelligent-Tiering, Glacier Instant Retrieval, and Glacier Deep Archive according to access frequency and retention needs.
How DHI Group accelerates generative AI workloads from idea to production using hackathons
DHI Group needed to move generative AI efforts beyond proofs of concept because its traditional software development lifecycle took months, while it sought faster delivery, technical validation, and broader AI literacy. AWS and DHI used a four-phase Hackathon Acceleration Package—preparation, enablement, a three-day build event, and a path to production—with workshops on Amazon Bedrock AgentCore, AI-DLC, and Kiro. Twenty participants formed three teams and produced prototypes for employer analytics, intelligent candidate matching, and a unified ClearanceJobs–AgileATS marketplace; DHI leadership committed to advancing all three toward production. The winning marketplace uses an MCP server, Amazon Bedrock AgentCore, Lambda functions, session memory, and Claude 3.5 Haiku to let recruiters search candidates and enrich profiles through natural-language requests. The post concludes that time-boxed, cross-functional hackathons can compress ideation, architecture, prototyping, executive alignment, and production planning when production criteria and preparation are established upfront.
Postgres week in the Netherlands: PGDay Lowlands & Percona Live 2026
An account of three days in the Netherlands covers talks at PGDay Lowlands in Utrecht and Percona Live 2026 in Amsterdam, including a five-minute presentation on ClickHouse integrations and a PostgreSQL 19 monitoring session. pg_clickhouse uses a foreign data wrapper to expose ClickHouse tables as PostgreSQL foreign tables, allowing existing queries, ORMs, and dashboards to continue working while pushable work runs in ClickHouse; 15 of 22 TPC-H queries at scale factor 1 are fully pushed down. pg_stat_ch captures per-query execution events in a shared-memory ring buffer and sends batches to ClickHouse, dropping events rather than blocking PostgreSQL when ClickHouse is unavailable. The monitoring talk covered PostgreSQL 19 logging, WAL and I/O statistics, WAIT FOR, new system views, and multixact and wraparound changes. The post also reports talks on optimizer hints and plan stabilization, and says both conferences will return in 2027.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.