Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,318 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
How we improved image download sizes on Medium with just four characters
Image-heavy websites need browsers to choose efficient image variants, but srcset and sizes can require complex breakpoint rules for responsive layouts. The post distinguishes picture, which lets authors prescribe sources based on conditions, from srcset, which gives browsers candidate dimensions and lets them choose using context such as connectivity and cache state. It explains that browsers begin fetching images before layout is rendered, while lazy-loaded images can defer fetching until their dimensions are known. With WebKit's March 24, 2026 support for auto in sizes on lazy-loaded images, the recommendation is to prepend auto to those sizes attributes; unsupported browsers ignore the unparseable rule. On Medium, the change made the home feed's image-related download chunk about 7% smaller on a small, but not mobile-small, display, and the text says WordPress updated every instance broadly.
Agents can do the work
Agent evaluations often ask what an AI Agent can do, but the larger constraint may be whether an organization is structured to let it act. Across organizations in energy, healthcare, ecommerce, gaming, financial services, and property management, teams found that Agents could often perform actions technically, but stalled on unreliable identification, inaccessible live data, or unreachable vendor systems. The post describes five readiness types—content, scope, procedural, data, and execution—and says deeper automation depends on documented processes, trustworthy connected data, stable APIs, and clear ownership of risk. Teams that lacked these foundations often redesigned toward guiding users rather than executing changes, while teams with standardized operations could trigger tightly controlled actions. The unresolved question is how readiness develops, since the gaps may involve undocumented procedures, fragmented systems, missing APIs, or infrastructure and accountability investments beyond buying a more capable Agent.
Backstage with Lakebase, part 2
Part 2 shows how moving Backstage’s operational Postgres database to Databricks Lakebase brings security, auditing, masking, and cost controls under Unity Catalog. The proof of concept used Lakehouse Federation to expose the catalog as the lakebase_bs foreign catalog, replacing cross-service RDS investigations with Unity Catalog grants and system.access.audit records. Audit events identify OAuth users and source IPs, while billing attributed 31.6130 DBU to production and 0.0107 DBU to a dropped test branch. Branch-level attribute masking propagates automatically to feature, CI, and QA copies, while the post cites a Perforce report stating that 60% of organizations experienced breaches or theft in non-production environments. It also introduces LakebaseOps, with three agents and seven scheduled jobs, and Lakebase MCP, exposing 46 governed tools across four profiles, so DBAs can design policies and workflows instead of handling repetitive provisioning.
Operator: A look under the hood
Operator is a customer-operations Agent built to help teams understand, manage, and improve customer experience across workspaces. The post distinguishes a production Agent from a prototype that simply gives an LLM API access and a system prompt, describing more than 50 purpose-built tools and 10 skills for data retrieval, reporting, investigation, and operational tasks. Its intelligence layer combines semantic search tuned against millions of support conversations, workspace-specific attribute awareness, and encoded reasoning, while the action layer supports configuration and content changes through reviewable diffs requiring explicit approval. A hybrid conversational and graphical interface presents charts, dashboards, and structured proposals within the existing platform. The post argues that read-only custom tools can be built in weeks, but production reliability, safe actions, multi-tenant isolation, graceful degradation, and ongoing maintenance require substantially greater investment.
How the D. E. Shaw group powers high-cardinality observability at scale with ClickHouse
Inside D. E. Shaw group, a global investment and technology development firm, millions of compute workloads run monthly across its high-performance computing infrastructure. To support capacity planning and resource allocation, it collects task-level metrics for requested and utilized compute, producing millions of distinct time series that its previous observability platform could not accommodate. Engineers benchmarked databases on identical hardware and datasets; ClickHouse ingested approximately 3.5 million samples per second on one node versus roughly 480,000 for a competing solution, while similar queries completed in seconds instead of timing out. They retained InfluxDB line protocol, batched messages into ClickHouse, and built disk-based tooling to transform and stream historical data; the resulting four-server NVMe cluster stores about 68 TB compressed and sustains 530k records per second on average. Materialized views support capacity-planning dashboards, while OpenTelemetry tracing extends the platform to other observability workloads, with SQL migration and data modeling requiring engineering effort.
ClickHouse vs Prometheus for High Cardinality, Part 2: Cardinality in ClickHouse
Traditional time-series databases like Prometheus experience write amplification, memory overhead, and operational challenges when high-cardinality label combinations churn rapidly. ClickHouse alters these economics by abandoning dedicated series objects in favor of a wide events model, storing telemetry as rows in columnar tables with dynamic Map attributes and metric measurements. Dynamic attributes can be handled using sharded Map types alongside materialized columns and text indexes for frequently queried labels. This design trades ingestion overhead for query-time processing, relying on columnar compression, partition pruning, and parallel scans to compute aggregations on demand. ClickHouse avoids the ingest-side bottlenecks of high cardinality, though Prometheus remains preferable for moderate cardinality, native PromQL semantics, and single-series lookups.
From manual to autonomous: how AI agents are transforming electric grid operations
Electric utilities are facing rising demand, retiring generation, extreme weather, aging infrastructure, and fragmented operational data that manual processes cannot manage at scale. AI agents are presented as a human-centered alternative that synthesizes heterogeneous data, learns from outcomes, and progresses from human-approved recommendations to exception-based control and eventually autonomous operations within defined parameters. Hawaiian Electric used a Retrieval Augmented Generation proof-of-concept with Databricks AI Search, Unity Catalog, and Lakeflow Declarative Pipelines to query regulatory documents and provide page-specific citations. The system reduced response times from five minutes to five seconds and was implemented in two weeks, while the article describes broader potential for predictive maintenance, outage response, load forecasting, and customer service.
Beyond Prediction: Solving the Multiple Knapsack Problem at Scale: How Uber Optimizes Incentives
Uber’s Tarot platform frames incentive allocation as a Multiple Knapsack Problem (MKP), replacing FIFO contention handling and reactive budget management. It combines segmentation, budget pacing, uplift predictions, multi-objective value scoring, and a plug-and-play optimizer coordinated by an orchestrator. For each run, Tarot forms user-lever opportunities, predicts incremental ROI and cost, reconciles available budget, and selects assignments under team, business, and concurrency constraints before publishing them to domain services. The initial HiGHS linear programming approach took more than 24 hours to find a feasible solution for 100,000 users and one lever, while CP-SAT solved the same dataset in minutes and handled millions of users across multiple levers within stated SLA timeframes. The result is a production architecture that moves beyond heuristics by combining constraint programming with budget pacing, while future work targets automated exploration and model feedback.
ClickHouse vs Prometheus for High Cardinality, Part 1: Understanding the Problem
Observability systems frequently encounter high cardinality when metrics contain numerous unique label combinations across multidimensional cloud infrastructure. In Prometheus, every unique label combination instantiates an independent time series that incurs metadata overhead, requires in-memory memSeries registration, and continually expands inverted index posting lists. The ingestion engine stores samples in two-hour Head blocks using compressed 120-sample chunks with XOR-based encodings, but rapid series creation and elevated scrape frequencies multiply chunk allocations and memory consumption. When executing read operations, broad aggregations and regular expressions require intersecting expansive posting lists and decoding entire chunks sequentially because the engine cannot push value predicates directly into compressed storage blocks. Consequently, managing high-cardinality workloads in series-oriented architectures forces operational trade-offs between system stability and granular visibility into ephemeral components.
You can't whisper at an AI agent
Stripe engineers ran roughly a dozen experiments to determine how to get AI agents to use current API versions, follow integration best practices, and load the Stripe skill. Passive hints, including SDK files, AGENTS.md guidance, and API warning responses, were largely ignored, while active prompts and distribution mechanisms performed better. A modular, progressively disclosed skill structure outperformed a monolithic file by roughly 10% and reduced token usage; a CLI login prompt led 30–35% of viewers to copy its install command, and explicit compatibility-mode errors prompted agents to correct version mismatches. The results suggest that agent-facing developer experience is primarily a distribution problem, with hard steering through errors and loaded instructions more reliable than soft hints agents may never encounter.
Postgres FDW: Pushdown is a negotiation
Postgres Foreign Data Wrapper extensions allow PostgreSQL to query external datastores like ClickHouse by delegating execution to remote engines. Engineering pg_clickhouse centers on pushdown decisions that determine whether SQL expressions run remotely or stream raw data back across the wire. Because pushdown depends on planner callbacks, deparser translations, and ClickHouse semantic compatibility, a single untranslated clause can block upper-level query pushdown entirely. Resolving pushdown barriers enables complex analytic queries to return hundreds of rows in milliseconds rather than pulling tens of millions of rows for local processing. Ultimately, pushdown development requires an iterative negotiation across differing SQL grammars, occasionally requiring the revocation of translations that fail to maintain strict equivalence.
Data quality is the AI strategy
NYU Langone Health’s AI strategy starts with data quality, arguing that healthcare AI cannot be reliable when source data is fragmented or inconsistent. The institution standardized on common transactional platforms, including one electronic health record and one ERP system, established authoritative data sources, and fixes data at the source rather than mapping it in the warehouse layer. Its Databricks-based unified data and AI platform, with Unity Catalog, supports clinicians, analysts, scientists, and corporate users across care, operations, and research, while real-time feeds power emergency-room decision-support models. Mherabi also describes a three-layer analytics model: structured visualizations, conversational tools such as Genie, and answers delivered in formats suited to the user. The stated conclusion is that upstream data discipline, governance, literacy, and adaptable platforms provide the foundation for trustworthy AI and timely clinical insight.
ClickHouse Cloud: Fast, Updatable Lookups with the Join Table Engine
ClickHouse Cloud provides an in-memory Join table engine for precomputed hash-map lookups and joins, aimed at enriching append-only fact data with dimensions that change occasionally. With Join(ANY, LEFT, key), inserting another row for a key acts as an upsert, while joinGet and matching JOIN clauses use the in-memory structure. Open-source Join tables are not distributed and persist each INSERT batch as separate .bin files without automatic compaction, but ClickHouse Cloud transparently uses SharedJoin with MergeTree for ALL or ReplacingMergeTree for ANY. In the demonstrated enrichment query, duplicate inserts for id 1 leave value 42 visible, while the underlying table retains duplicates until merging; the in-memory load applies FINAL and always selects the latest entry, making ANY LEFT suitable for frequently updated dimensions.
Lessons from Building a First-Pass AI PRD Reviewer at Uber
Uber built an AI-powered PRD Evaluator to help product managers strengthen drafts before they enter structured checkpoint and approval forums. Starting with a PRD, it gathers linked documents, decks, meeting notes, prior experiments, cross-functional artifacts, and Uber-specific context, then classifies the proposal to calibrate review depth. It assesses opportunity and hypothesis, product scope, user experience and impact, and metric and data rigor, producing a launch-readiness scorecard with prioritized gaps, evidence, replacement-text suggestions, and a “start here” fix. Early internal use by dozens of PMs helped surface blind spots, test unsupported headroom assumptions, identify adjacent-system effects, and improve experiences within existing scope. The team’s lessons emphasize decision frameworks, broader context, hard boundaries, prioritization, and preserving human judgment for final approval.
Building Blocks for Foundation Model Training and Inference on AWS
The post explains how foundation-model scaling now spans pre-training, post-training, and test-time compute, creating shared infrastructure requirements rather than a single pre-training curve. It examines how AWS infrastructure and open-source tools support these workloads across accelerated compute, networking, storage, orchestration, ML frameworks, and observability. Technical detail includes NVIDIA H100, H200, B200, and B300 GPU characteristics, NVLink/NVSwitch scale-up communication, and EFA-based scale-out networking for distributed collectives. The architecture also incorporates EC2 P instances, tiered storage, Slurm, Kubernetes, SageMaker HyperPod, CUDA, NCCL, PyTorch, Prometheus, and Grafana. The conclusion presents integration points between these layers as the basis for diagnosing bottlenecks and making scaling decisions across the foundation-model lifecycle.
Beyond efficiency: PayPal expands what's possible to build with AI
PayPal describes how it used Cursor to accelerate software development across an organization of 8,000 developers and a codebase spanning decades. It began with high-impact teams because the company moves money for more than 400 million customers and mistakes are costly; within two weeks, those teams were deploying daily rather than weekly or biweekly, and adoption spread organically. The company says a Java upgrade across 3,000 applications took two months instead of an estimated eight to twelve, while highly adopting teams saw more frequent deployments, shorter lead times, and lower change failure rates. PayPal continues to measure deployment frequency, lead time, and change failure rate rather than AI-generated code, and Michelle Chance believes the company could deliver 40% more capabilities in 2026 than in 2025.
Predictive quality starts where defect detection stops
Manufacturing quality teams often receive defect-rate reports after the conditions behind them have changed, because inspection, supplier, and environmental data are disconnected. Predictive quality combines production, inspection, and supplier data with machine learning to forecast defects before final inspection, shifting quality management from reactive documentation to proactive intervention. Databricks Genie is presented as a natural-language interface for querying those sources together, including questions about first-pass yield, supplier lots, root-cause contributors, and process conditions; its answers include citations and can surface unusual patterns. The described capabilities include contextual understanding of terms such as NCR, CAPA, and CPK threshold, multi-source reasoning, and traceable outputs tied to records. The intended outcome is faster analysis and earlier action to reduce scrap before its cost is incurred.
ML based ranking using Nrtsearch
Nrtsearch’s Inference Plugin embeds machine-learning ranking in the Lucene-based search engine, addressing scalability problems in a two-stage design where a separate inference service scored candidates. Previously, large candidate sets and feature payloads created network-transfer and serialization overhead; the new flow extracts features from index documents or request parameters and scores documents within replica nodes. An ML Scorer selects models, transforms features, runs MLeap-based inference, and aggregates scores, with support for XGBoost, neural networks, multi-model execution, batch inference, and custom Java scorers. Models are loaded from MLflow through cluster configuration, while teams can use Lucene expressions or implement modular interfaces; testing, canary deployment, rolling updates, and Prometheus metrics support operations, and future work includes GPU-based ranking and additional inference engines.
Operating room utilization is hiding in your scheduling data
Operating room utilization measures in-room surgical minutes against allocated block-time minutes, yet most US health systems reportedly run at 65–75% versus an 80% industry target. Daily performance reports arrive the next morning, after schedules are set and opportunities to release unused blocks, redeploy staff, or backfill add-on cases may have passed. The post presents Databricks Genie as a natural-language interface for querying scheduling, utilization, and outcomes data without a data analyst request. Its proposed analytical environment combines scheduling data, actual case logs, block-release records, contribution margin, and staffing costs, with breakdowns by surgeon, service line, facility, and day of week. Genie surfaces specific intervention targets, although the post says it provides data access rather than automating OR management.
Growth analytics is what comes after growth hacking
Growth analytics is presented as the next stage after growth hacking, arguing that competitive acquisition now depends on precise unit economics, cohort quality, and rapid conversion optimization rather than isolated tactics. It distinguishes product analytics, which explains feature use and user flows, from growth analytics, which connects acquisition sources and costs with activation, revenue, retention, and cohort behavior. The stated bottleneck is fragmented tooling: answering questions such as 90-day LTV by channel correlated with seven-day activation can require manually joining multiple systems and delay weekly budget decisions. Databricks Genie is described as a governed natural-language interface over unified acquisition, behavioral, and billing data, supporting attribution-to-LTV analysis, payback modeling, and paid-versus-organic comparisons. The text reports a 50% relative acquisition-rate lift, from 8% to 12%, for customers using Genie, and shorter onboarding insight cycles from months to weeks.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.