Loading…
Browse by type
Problem & solution reads
Problem & solution posts on Yomu, summarised newest first.
- 891 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Problem & solution posts
Automated merchant review summary system with integrated feedback
The system uses representative sampling, large language model (LLM) summarization, automated quality checks, and consumer feedback to generate concise summaries from written merchant reviews. Consumers can scan an overall summary or focus on specific topics such as taste, packaging, and portion size, while merchants can identify recurring themes without reading every comment. The pipeline calculates the merchant’s rating distribution, uses a sentiment model to score reviews, and applies stratified sampling to select a representative subset before sending them to an LLM for summary generation. A separate LLM validates that each claim in the generated summary is supported by the source reviews, and summaries that fail this automated assessment are withheld from publication to reduce unsupported claims. The system regenerates summaries when new reviews or negative consumer feedback reach configured thresholds, and the feedback-driven regeneration includes anti-abuse controls to maintain trustworthiness.
How Intelligent Load Shedding Prevents Cascading Failures in Tier-0 Systems
Salesforce’s Cloud Atlas team operates a globally distributed identity data store that must maintain five nines of availability, roughly five minutes of downtime per year. The team redesigned service protection after per-instance rate limiting proved insufficient under autoscaling and increasingly bursty, unpredictable agent-driven traffic. The new approach combines intelligent load shedding, which monitors queue time as an early indicator of system pressure before error rates spike, with coordinator-free global quota management. Servers independently measure and share customer usage to throttle noisy tenants without a centralized coordinator, preventing one customer’s surge from affecting others. This design prevents cascading failures by shedding lower-priority work before overload spreads across dependent services, ensuring graceful degradation rather than widespread outage.
Designing the Kotlin Multiplatform and TeamCity Integration
The post describes the design of a new TeamCity integration for Kotlin Multiplatform (KMP) developers building iOS apps. The integration addresses common pain points: iOS builds require macOS infrastructure, Apple signing is complex, and there is no guided path from IDE to CI/CD. The solution provides a progressive onboarding flow within the IDE, starting with a tooltip after the first Git push. The flow generates four configuration files (teamcity.yaml, Fastfile, Appfile, Gemfile) and runs a build and tests on a hosted macOS agent, then guides the developer through a single form for Apple signing and TestFlight publishing, and finally automates the pipeline on GitHub pushes. TeamCity Cloud provides hosted macOS agents and a free starting tier, so no Mac is required. The design was validated through usability sessions and focuses on meeting the developer at the right moment with minimal configuration.
How I built agent-based security reviews on Databricks
Security reviews at Databricks relied on automation but still consumed expert time on predictable work, leaving less room for genuinely novel, high-risk architectures. The author built an agent-based layer on Databricks using Unity Catalog for governed tables containing standards, request data, and decisions; Databricks-hosted foundation models (Claude Haiku, Sonnet, Opus) for classification and reasoning; and Lakeflow Jobs for orchestrating notebook-based workflows on serverless compute. A conversational intake app built with Databricks Apps turns plain-language descriptions into structured requests and provides preliminary risk indications before a formal request is created. Seven focused agents each have a bounded responsibility—intake, risk assessment, requirements, specialized reviews, validation, workflow, and learning—ensuring behavior remains inspectable and testable. Routine requests that once waited days in the queue can now be completed in minutes, and security engineers spend more time on designs requiring genuine expertise.
Hitting a billion tokens per minute on one GPU by combining a query planner and an inference engine
Modal and CMU built Quail, a query-aware inference layer for AI-SQL that combines a query planner with an inference engine to process structured LLM transformations of database entries. On a multi-join query, Quail achieves over a billion tokens per minute per H100 GPU, over 10x faster than vLLM, costing under 6 cents per billion tokens on Modal. The gain comes from using the query plan to order requests for better KV cache management, including a revised Hydragen-style cascade attention and batching small requests to reduce host overhead. On a new AI-SQL benchmark, Quail runs 1.84x faster than vLLM, but falls behind on agent trace processing. The authors see this as part of a trend toward custom inference engines for specific workloads, with a reusable core like databases post-DataFusion, and note that much work was done by prompting coding agents.
A Practical Recipe for Semi-Supervised Federated ASR: Online Pseudo-Labels with Server Update Stabilization
Semi-supervised federated learning (SSFL) for ASR uses a teacher to generate pseudo-labels on unlabeled client data while the server trains on a small labeled seed. The paper finds that pseudo-label errors compound across the output sequence and training rounds, causing divergence and a large gap to fully-supervised FL. Two coupled design axes close this gap: the teacher type (per-client online, global broadcast, or transitioning) and the anchor (server-side labeled-data updates that stabilize training). A per-client online teacher diverges on its own but, once stabilized, matches or beats the global teacher in-domain and under domain shift. The server must keep training on labeled data between rounds; otherwise the online teacher drifts, and this interleaving governs convergence more than the seed model does. The proposed guidelines improve over the strongest prior method on 9 of 11 pairs, by 20.8% in-domain and 10.0% cross-domain on average.
Reproducing OLMo 3 7B Pre-training in MaxText: case study of large scale training on TPUs
The post describes the reproduction of AI2’s OLMo 3 7B pre-training using MaxText on Google Cloud TPUs, covering both stages, matching the reference loss curve and held-out metrics. PyTorch weights were converted to JAX with logit parity, KL divergence 1.5e-3 and 98.75% top-1 agreement, and held-out evaluations caught a data loader bug that made incorrect gains appear. The run demonstrated reliability via exact checkpoint-resume with zero loss delta, resilience to a 75% capacity loss via resize with 1% throughput preservation, and seamless TPU generation change. Performance optimizations achieved 44.5% MFU on Ironwood, and head reshaping from 32 heads by 128 to 16 by 256 yielded 12.4% faster throughput at identical parameters and FLOPs. The post emphasizes that faithful reproduction requires verification on generalization metrics and provides instructions for readers to reproduce the run with open sourced code.
Compressing Streaming Neural Audio Encoders via Latent-Space Distillation
Apple’s on-device dictation tokenizer maps waveform to latent representations for a sparsely activated language model, and compressing it reduces memory and latency demands. The authors compress it via latent-space distillation, using the teacher’s pre-quantizer latent as the supervision target rather than tokens or output distributions. A student encoder is trained to regress the teacher’s per-frame latent with a squared-error objective, and an affine layer absorbs width mismatch. This recipe works for both token interfaces and for tokenizers pretrained alone or jointly with a language model, requiring no fine-tuning. At 2.8x compression, the student stays within 1.9% relative WER of its teacher on five of six pairs and outperforms an independently trained tokenizer of identical capacity by 3.9% relative.
Rendering huge pull requests in the GitHub Copilot app
Rendering a pull request with over a million changed lines and hundreds of threaded comments requires a fundamentally different architecture than a code-only diff. The standard virtualized approach fails because comment heights depend on render-time factors like markdown wrapping, expandable sections, and reply composers. The team split the document’s height into two independent domains: deterministic code geometry and dynamic block geometry for comments. A single idle- and scroll-gated measurement pass, scoped to the viewport, corrects heights without causing reflows mid-scroll. The result is a pull request view where a million-line diff with hundreds of comments opens, scrolls, and behaves like a normally sized pull request.
Manage Kubernetes Node Fleets with NodeWright
Managing Kubernetes node fleets, especially GPU nodes, is difficult because host-level changes can disrupt long-running training jobs, and traditional tools are not cluster-aware. NodeWright, an open-source Kubernetes-native package manager from NVIDIA, solves this by declaratively configuring and updating node operating systems while respecting PodDisruptionBudgets and non-interruptible workload labels. The operator orchestrates a sequence of cordon, wait, drain, apply, interrupt, and uncordon on each node, using container image packages that carry modifications such as kernel parameters, security agents, and verification scripts. It supports progressive rollouts across a fleet via DeploymentPolicy compartments with configurable disruption budgets and strategies like batch, canary, and rolling. NodeWright integrates with NVIDIA AI Cluster Runtime for validated configurations and is part of the DSX OS open-source portfolio, available under Apache 2.0.
How I massively improved my AI inference performance without buying new hardware
Default inference configurations for large language models often leave significant performance unused, even on expensive hardware like 16 NVIDIA H200 GPUs across two nodes. The author tested this setup with vLLM and Red Hat AI Inference 3.4 on a CoreWeave Kubernetes cluster, using enterprise RAG workloads with 2,000-token inputs and 100 concurrent users. By systematically comparing aggregated and prefill-decode (PD) disaggregated architectures, and tuning tensor parallelism (TP) and PD split, latency was dramatically reduced. For Qwen3-32B, time-to-first-token (TTFT) on the best aggregated configuration (2xTP8) dropped from 995ms to 304ms, and with optimized routing to 287ms — a 71% improvement without hardware changes. For Llama-70B, TTFT fell from 1,292ms to 463ms, a 64% gain. The author also built ServeIt Studio, an open-source tool that automates the full optimization pipeline on Kubernetes.
How to test that a dozen MongoDB drivers, in a dozen languages, conform to a dozen specs
MongoDB develops a dozen native client library drivers in a dozen languages, each idiomatic to its language but needing consistent API behavior. Instead of relying on English specs or Cucumber tests, engineers created a YAML-based domain-specific test language interpreted by runners in each driver. The Unified Test Format, built on this YAML language, now contains nearly 1,500 files testing conformance to specs covering retries, transactions, client-side encryption, and more. A natural experiment showed four of five drivers had lower nonconformance bug rates after adopting YAML tests; Node.js improved after fixing its test runner. The IEEE ISSRE 2026 accepted a paper on this approach, which has saved over 22,000 lines of test-runner code across seven drivers.
Collected, stored, and useless: The lifecycle of most customer data
The post examines the typical lifecycle of customer data in companies, revealing that most data is captured but never effectively used. A survey of 660 business leaders found that 48% struggle to connect customer data across channels, making it the top obstacle over budget or headcount. Data is often captured in volume but siloed across systems, duplicated into separate records for the same customer, and reliant on stale overnight batch updates. This leads to AI agents working with outdated profiles, causing 71% of consumers to abandon conversations when the AI fails to recognize them. The post proposes shortening the path between capture and use through real-time data synchronization, exemplified by Twilio’s Conversations and Conversation Memory products.
The 2026 CDP Shift: Why AI Agents Are Only as Smart as Your Data Infrastructure
The 2026 Customer Data Platform Report identifies a critical enterprise challenge: while AI tools grow individually smarter, fragmented customer data causes a worsening, not better, experience. The shift is from computational capacity to contextual continuity, as disconnected pipelines force AI agents to start every interaction from scratch. Three failures arise from fragmented systems: agent amnesia, escalating operational costs from wasted tokens, and trapped intelligence that never reaches active workflows. Modern CDPs have evolved from passive audience builders into API-first context engines that support three pillars: pre-execution context with working memory, unified behavioral and conversational data streams, and infrastructure-level governance enforcing compliance before data reaches AI models. Organizations like Twilio, Trustpilot, and Quartz demonstrate that embedding real-time customer context into the conversation layer outperforms deploying the smartest standalone models.
How to design an AI-to-human handoff that preserves context
AI-to-human handoffs often fail because context is lost during transfer, forcing customers to repeat themselves. 76% of consumers report that human agents have little to no context about their issue, and 91% of brands have seen customer satisfaction drop due to AI missteps, with 40% attributed to context loss between channels. A cold transfer drops the customer into a new queue with nothing carried over, while a warm transfer briefs the receiving agent beforehand. Five steps to preserve context include defining escalation triggers early, specifying a payload with conversation history and customer profile, generating a three-line summary, maintaining context across channel changes, and writing back learned information to customer records. Measuring repeat-explanation rate and post-handoff resolution rate helps identify whether handoffs are working or if the AI is over-holding conversations.
How Stripe is designing Checkout for AI agents
Stripe implemented WebMCP on its Checkout UIs, which processes around 0.45% of global GDP, to optimize the buying experience for AI agents. Previously, agents relied on slow browser automation, consuming an average of 1.8 million tokens and 38.83 tool calls per transaction, taking over 2.5 minutes. WebMCP exposes MCP tools from the client side, allowing agents to execute actions like get_order_summary or fill_payment_form instead of interpreting the DOM. The implementation uses progressive tool disclosure, revealing only relevant tools; in evaluations across six models, agents using WebMCP consumed 42% fewer tokens, made 38% fewer tool calls, and completed checkout 39% faster. Stripe reused existing UI state and forms via imperative and declarative APIs, avoiding a separate agent-only code path and keeping human and agent experiences aligned.
Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem
Removing entire transformer blocks is a cheap way to speed up large language models, yielding predictable inference gains and stacking with quantization and other compression methods. The challenge is that block choices interact, making it a combinatorial problem rather than a simple ranking task. The authors reformulate block selection as a constrained binary optimization (CBO) problem, mapping it onto an Ising glass, a disordered spin system with all-to-all interactions and a fixed number of removed blocks. They note that mean-field methods, which treat each block independently, are inadequate for this interacting system. Block removal composes with quantization, low-rank compression, width pruning, and knowledge-distillation-based healing, fitting into a larger pipeline rather than competing with it. The paper is titled ‘LLM Compression by Block Removal with Constrained Binary Optimization.’
More data won't make your AI smarter (here's what will)
Twilio research found that 99% of companies report something blocks them from understanding customers, with 48% citing connecting data across channels as the top obstacle. The problem isn’t data collection, which has succeeded, but the missing data activation layer that makes context usable in the eight seconds an AI agent has before a customer gives up. Twilio’s solution involves three elements: continuity across channels, real-time profile freshness, and retrieval of relevant history mid-conversation. Twilio Conversations treats touchpoints as a single lifecycle, Conversation Memory syncs new details in real time, and Conversation Intelligence surfaces approved documents alongside history. Twilio recommends starting small by picking one handoff where customers repeat themselves and making context survive it, rather than feeding the AI more data.
Financial Crime Mitigation with MongoDB / Series Overview
Financial crime losses could exceed $2 trillion annually, and legacy systems—fragmented, batch-processed, and transaction-focused—cannot keep pace with modern, AI-powered attacks. MongoDB Atlas offers a unified platform that consolidates entity management, transaction monitoring, behavioral risk scoring, model management, and case management on a single flexible data foundation. Key capabilities include AI-driven entity resolution via Atlas Search’s $rankFusion, network analysis with $graphLookup, and hybrid detection combining rules with vector search. The platform enables real-time intelligence through Change Streams and Time Series Collections, allowing immediate action at the moment of decision instead of retrospective analysis. Organizations can either buy off-the-shelf solutions or build custom platforms on MongoDB, with the build approach offering rapid iteration, freedom from vendor constraints, and competitive differentiation.
RADAR: Catch gray failures with anomaly detection
Gray failures are partial customer-facing problems that can leave health checks green while a subset of users quietly fails. RADAR (Reliability Anomaly Detection, Alerting, and Root-cause analysis) addresses this gap by collecting error counts and distinct affected users, segmented by error code and region, then applying the unsupervised streaming SPOT model. SPOT learns normal behavior from the previous 14 days using a single risk parameter, while downstream stages enrich, filter, deduplicate, and route alerts before providing anomaly details and AI/BI Genie-backed dashboards for investigation. Running RADAR internally reduced incident-discovery time by 95% with over 90% precision and no human needed to spot the pattern. The post also describes a Databricks scaffold that maps metrics, detection, alerting, and root-cause analysis to platform components and can be assembled by an AI agent from a metric, scaffold, and prompt.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.