Loading…
Browse by type
Problem & solution reads
Problem & solution posts on Yomu, summarised newest first.
- 928 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Problem & solution posts
Reading the signals: Helping your AI-built payment integration understand business intent
AI agents can quickly produce Stripe payment flows that create a PaymentIntent, render the Payment Element, and pass an end-to-end demo, yet a correct implementation can still violate the business intent behind the money. The post argues that payment expertise and business decisions are often missing from integrations built from API references, causing silent failures such as terminal declines being retried, intents stuck in requires_action, uncaptured authorizations, restricted connected accounts, and asynchronous payments being refunded. It recommends defining business intent before configuring the PaymentIntent, then writing those rules into code comments, metadata, documented requirements, or custom objects. A verifier agent can check rules on every build, either through an agent skill, automated tests, or CI, while shared ownership remains with the human who decides what correct means.
How Scottish Water Made Its Capital Investment Data Conversational With Databricks Genie
Scottish Water’s Capital Investment teams had project data, reports, and metrics available, but non-technical users often struggled to find answers, creating duplicated reporting and delays. SPARK addresses this access problem through a natural-language interface in Microsoft Teams: Copilot routes questions via a supervisor agent and MCP to the Databricks Genie Agent, which queries governed Unity Catalog data. The implementation curates use-case data in a gold layer, adds metric views as a semantic layer, and configures business rules, examples, benchmarking, feedback, and monitoring for accuracy and performance. Environment-parameterised Databricks Asset Bundles, separate environments, Azure DevOps approvals, Entra ID, and Key Vault support repeatable deployment. The post reports lookup journeys shrinking from around eight clicks or four to five search steps to a direct question, with an estimated 520 to 1,300 hours saved annually under a stated usage scenario.
Certificate Transparency Monitoring is now generally available
Cloudflare announced the general availability of Certificate Transparency Monitoring alongside a filtering mechanism to eliminate noisy alert emails. Previously, the monitoring system alerted domain owners for all Certificate Transparency log entries, including routine, automated renewals for Universal SSL, Advanced Certificate Manager, and backup certificates. Because the alerting flow and certificate ordering service operated independently, existing identifiers like TBSCertificate hashes arrived too late to verify pre-certificates. To resolve this race condition, the ordering service now computes an SHA-256 hash of the DER-encoded SubjectPublicKeyInfo structure at key generation. When the alerting service discovers a log entry, it recomputes this hash to verify ownership against the ordering database, suppressing alerts for Cloudflare-managed certificates while preserving alerts for external or unexpected issuances.
Automated agent triage with Agent Tracing and Claude Routines
Evaluating Sentry's AI agent Seer previously required manually querying evaluation data and parsing raw JSON traces to find tool failures, latency spikes, and hallucinations. To automate this triage, the team deployed a daily Claude Routine connected to Sentry using a Model Context Protocol server. Each morning, the routine analyzes hundreds of overnight agent conversations, reviews aggregate error metrics, samples failed and successful conversation spans, and inspects the agent codebase for context. Claude evaluates whether final decisions match underlying reasoning and creates Linear tickets for newly discovered issues. This daily automated analysis accelerated the feedback loop, allowing engineers to identify efficiency issues like repeated malformed repository queries and deploy same-day fixes.
Reducing Text2SQL latency with parameterized query templates
In Text2SQL systems, generating SQL on the fly with frontier foundation models can cause response times of 25 to 30 seconds due to 60,000-token prompts and model latency. To solve this without degrading accuracy, an intelligent caching layer was built using parameterized SQL query templates paired with question embeddings. When an incoming question matches a cached template via semantic search above a confidence threshold, named entity recognition extracts values to populate template placeholders, bypassing the initial SQL generation step. In production, this approach reached a 60% cache hit rate after two weeks, cutting per-hit token consumption by roughly 90% and reducing overall latency by 80% to under 5 seconds.
Adobe Firefly: Simplified observability with Amazon Managed Prometheus
Adobe Firefly powers generative creative features using a large-scale GPU training infrastructure hosted on Amazon Elastic Kubernetes Service. As workloads scaled to thousands of nodes and GPUs, Adobe's self-managed Prometheus setup struggled with query latency across high-cardinality telemetry data. To resolve these performance bottlenecks, the team moved two million critical time series metrics to Amazon Managed Service for Prometheus while using managed scrapers alongside their existing infrastructure. This setup accelerated query response times by more than 28x over 24-hour windows, reducing query execution times to approximately ten seconds. Additionally, the migration expanded practical monitoring windows from six hours to 24 hours, giving teams reliable visibility across long-running distributed training jobs.
Databricks Network Configuration delivery to Tens of Millions of Serverless VMs
Databricks’ serverless platform launches tens of millions of VMs daily, and each VM needs network configuration before serving customer workloads. The original design synchronously aggregated data from multiple upstream services on the cluster-creation critical path, producing 5,000-ms p99 RPC latency and availability bottlenecks as request volume reached billions per day. The replacement uses change events through a message queue, asynchronous per-workspace recomputation, partition-local snapshots, and periodic reconciliation to recover from missed events. The serving path now performs a single snapshot-store read without upstream calls, while events carry workspace and resource identifiers rather than customer data. After rollout, p99 RPC latency fell to 125 ms, server success rate rose to 99.99%, upstream call volume dropped 86%, and the legacy synchronous framework was deprecated.
How a major freight railroad scaled pipeline creation with Genie Code
A major Canadian freight railroad with roughly 20,000 route miles needed to modernize a decades-old analytical estate while handling hundreds of pipelines and rising demand for real-time analytics and AI. Using Databricks Genie Code, Unity Catalog, custom Agent Skills and a Streamlit Databricks App, the team created a metadata-grounded pipeline-generation workflow driven by compact YAML prompts. The workflow discovers schemas, maps source fields, and emits six artifacts—DDL, historical and streaming ingestion, incremental merges, and automated tests—using deterministic patterns for audit columns, deduplication, change-sequence guards and soft-delete reconciliation. Human-reviewed Source-to-Target Mapping preserves business interpretation, while Genie Code handles discovery, orchestration and artifact generation within governed Databricks execution. The reported result was more than 90% automation for new table ingestion, reducing delivery from days per table to minutes and supporting single, multi-table and bulk generation.
How iFood built its agentic security platform on ClickHouse Cloud
iFood’s security incident response team needed high-volume log ingestion, long-term retention, and rapid access to months of history, but its Databricks-based platform became costly to scale beyond 130 TB and imposed query timeouts. The team ran a production-data proof of concept and rebuilt its tooling around ClickHouse Cloud on AWS: logs land in S3 and flow through ClickPipes into ClickHouse, with Querybook preserving notebook-based investigations and ClickStack planned for operational metrics. With ClickHouse Cloud, iFood reports 9–16x faster queries, expected cost reductions of 40–50%, and data freshness improving from hourly batches to 2–10 minutes; alerts that previously ran hourly can now run about every 10 minutes. That foundation also supports agentic threat hunting: multiple sub-agents query more than 30 TB in parallel, reducing a hunt from about a week to roughly two hours, while Langfuse is used to inspect and tune agent analyses.
Running Cost-Efficient Export Workloads at Uber
Uber’s offline export workloads often retrieve a small set of records from large, date-partitioned Apache Hudi tables, but narrow historical predicates can still trigger full-table scans. Repeatedly touching files, metadata, or Parquet footers keeps Google Cloud Storage objects hot, undermining Auto-Class tiering and increasing storage, retrieval, metadata-operation, egress, and latency costs. The approach combines Hudi column stats with sorting on predicate columns: metadata-table statistics prune files without opening them, while clustered records tighten file-level min/max ranges and make pruning more selective. A Hudi secondary index was evaluated but did not reduce file scans enough because records for a subject were spread across most files. The conclusion is that a one-time rewrite and ongoing Hudi table services can turn broad historical scans into lower-touch lookups, while future work targets multi-column layouts and automated ROI-based optimization.
How we raised mobile end-to-end test stability to 98%
Shopify’s largest mobile app had become blocked by flaky end-to-end tests, which failed when screens took longer to render and had been removed from pull-request checks. Since 2023, tests used Appium through WebdriverIO and React Native Test IDs, allowing timing shortcuts and assertions about component-tree presence rather than what merchants could see or use. The rebuild hides Appium behind a strict builder-style wrapper that requires an assertion after every action; computer vision uses screenshots, PaddleOCR for text, and OpenCV for icons, with Test IDs as an opt-in fallback. After promotion into blocking CI, stability reached 98% of individual test successes over total runs, up from 50% with the old API, on two platforms. A pre-promotion flakiness gate rejects tests that exceed a failure threshold across repeated runs, while remaining failures largely involve network problems or simulator boot failures.
How Standardizing Product Telemetry Reduced Time to Insight by 97%
Salesforce faced scaling bottlenecks when individual engineering teams maintained fragmented, custom telemetry pipelines requiring manual data transformations and dashboard creation every release. To resolve these silos, the engineering organization established the Product Data Platform (PDP) using a standardized custom schema built atop Monitoring Cloud infrastructure. This framework mandates core telemetry fields for consistent metric generation while offering governed optional attributes for product-specific flexibility. An AI-driven Model Context Protocol tool assists developers by recommending compliant instrumentation patterns directly within their workflows. Today, the unified platform processes 45 billion rows of data daily across 19,000 distinct events and 2,000 product features, reducing dashboard refresh latency by 97 percent from one month to daily updates.
Enabling Seamless Kafka Async Queuing with Consumer Proxy
Uber’s Kafka deployment grew from 1 million to 12 million messages per second, but using Kafka’s ordered, partition-bound consumer model for asynchronous queueing created partition-scalability limits and head-of-line blocking. A one-second RPC example would require 1,000 partitions to reach 1,000 events per second, while non-uniform latency and poison-pill messages could stall unrelated work. Consumer Proxy addresses the mismatch by fetching messages through Kafka’s binary protocol, pushing them individually over gRPC, collecting processing statuses, and committing offsets only when safe. It also provides features including out-of-order commits, dead-letter-queue operations, flow control, and circuit breaking while hiding Kafka consumer-group management from services. At Uber, Consumer Proxy became the primary async-queueing solution, with hundreds of services onboarded; adaptive flow control and Kafka cluster federation are planned next steps.
Innocent until combined: Blocking the lethal trifecta with Omnigent Contextual Policies
An AI agent can exfiltrate private data when one session combines access to confidential information, attacker-controlled content, and an external communication channel, even though each capability is individually authorized. Omnigent's contextual policy tracks those three legs as session state: private data, untrusted content, and exfiltration, with human-defined tool or argument-based classifications. It allows ordinary actions until both prerequisite legs are lit, then denies the outbound call that would complete the trifecta. In the support assistant example, a user-supplied ticket instructs the agent to read internal Q3 revenue figures and email them externally; without the policy, the confidential number leaves in a reply, while the protected run blocks that email. A password-reset ticket still proceeds because it lights only the untrusted-content leg, and an empty internal lookup does not light the private-data leg.
Rebuilding Notion’s lexical search reindexer
Notion’s lexical search reindexer had become a multi-week, manually monitored process that produced about 90% data consistency. The old design also relied on Snowflake metadata lookups and a separate catchup pipeline. The redesign replaces ECS/Fargate workers and live Elasticsearch writes with an Apache Spark–native pipeline that routes documents to exact index-and-shard pairs and generates native snapshot files through embedded Elasticsearch nodes. Temporary indices capture live updates while the snapshot is built, after which aliases, external document versions, Elasticsearch’s _reindex, and phantom detection handle catchup, including alive-to-deleted transitions. The new system completes full reindexing in under two days, catchup in under an hour, reaches 100% consistency, removes Snowflake and ECS from the indexing path, and reduces manual intervention to under two hours per run.
Making Knowledge Distillation Cheap Enough to Run at Scale
Knowledge distillation trains a smaller student to match a larger teacher's capabilities, but conventional online training keeps both models loaded and materializes full vocabulary distributions at every step, making long-context recovery expensive. The proposed system caches the teacher's top-100 logits once and introduces a fused, chunked KL-divergence loss that avoids storing the full vocabulary-by-sequence grid or the student's full logits. In benchmarks with an 8K-token context on one H200, offline fused chunked KL used 58.3GB peak memory versus 102.8GB for online training, while all four methods reached near-identical training loss. At 32K tokens, the fully chunked loss reduced isolated kernel memory from 85.2 GiB to 5.45 GiB; distilling GPT-OSS 20B at that context cut step time from 57.0 to 12.23 seconds and reduced the setup from four GPU nodes to one.
From charging a blended rate to IC++: pass network costs to your connected accounts
Platforms using blended payment rates absorb interchange variance, which can reduce margins as card mix shifts toward premium rewards, corporate, and other high-interchange cards. Network cost passthrough, also called IC++, lets eligible Connect platforms pass interchange, scheme fees, and applicable adjustments to connected accounts at cost while keeping Stripe fees and platform markup separate. Eligibility requires IC+ pricing, platform-controlled pricing, enabled transfers, an appropriate Merchant of Record fund flow, and eligible geographies; passthrough can then be enabled in the Dashboard or through the API. The rollout also requires lower application fees, new network-cost reporting, and a pilot with clear communication before broader deployment.
How Musinsa scaled its audience engine with ClickHouse Cloud and reduced TCO by 71.4%
Musinsa’s Audience Engine powers a customer data platform that targets audiences across 16.4 million members. It stores each user-to-audience relationship as a separate row, producing roughly 1.1 billion cohort-to-user mappings across 2,500 audiences and potentially 16 million new rows when one audience is added. Its self-hosted ClickHouse deployment coupled storage and compute, forced workloads to share a cluster, and created operational burdens as data and query demands grew. Musinsa migrated to ClickHouse Cloud on AWS, using separated S3-backed storage, workload-specific compute, automatic idling, vertical autoscaling, and ClickPipes for direct real-time ingestion. The company reports an 86.5% reduction in storage costs and a total cost of ownership reduction of up to 71.4%, while shifting engineering effort toward CDP business logic.
How We Moved 1,500 Android Screenshot Tests to Roborazzi
Thumbtack migrated 1,500 Android screenshot tests from Firebase Test Lab to Roborazzi to eliminate slow and error-prone testing on remote emulators. The engineering team ported test suites from androidTest to test in batches of 20, adjusting test rules and annotations for Robolectric. Because Roborazzi runs in a single JVM process with parallel execution, the team had to debug shared font cache pollution and cross-platform shadow rendering inconsistencies between macOS and Debian. They resolved configuration repetition by mapping Robolectric annotations into module-level robolectric.properties files and avoided stale cached images by clearing Roborazzi's cache via a custom Gradle task. The migration significantly decreased CI runtimes while making local test recording and verification easier.
How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC…
Netflix designed a query execution serving layer for its Real-Time Distributed Graph to power sub-100ms responses across diverse graph traversal workloads spanning billions of nodes and edges. To prevent compounding network delays in multi-hop queries, the engine uses a breadth-first traversal model that batches entity lookups across entire frontier levels rather than tracing paths depth-first. The entire serving architecture relies on asynchronous composition across small thread pools of 16 to 24 threads, ensuring no thread blocks while waiting on remote storage or enrichment I/O. Selective caching via EVCache targets stable properties with volatility-matched TTLs, producing 70 to 80 percent cache hit rates and reducing backend storage calls by three to four times. A layered filtering hierarchy pushes depth and edge limits directly to the storage tier, eliminating bespoke code changes while isolating system resources against excessive fan-out.
Top topics
- Architecture834
- Performance816
- LLMs669
- Machine Learning622
- Open Source567
- AI Agents560
- AI480
- Developer Experience448
Weekly digest
The best engineering reads, once a week.