Loading…
Browse by type
Problem & solution reads
Problem & solution posts on Yomu, summarised newest first.
- 891 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Problem & solution posts
Application Metrics caught my broken size estimator
Cliparr’s video size estimator was intended to predict output size from target bitrate multiplied by clip duration, but real-world results could vary because its VBR encoder changes bitrate with content. Rather than manually testing a matrix of formats, codecs, and quality settings, the author extracted the browser-based export engine into Cliparr Convert, a standalone static converter, and instrumented it with anonymous Sentry Application Metrics. Completed conversions record actual and estimated bytes, their ratio and delta, plus attributes such as output format, quality, and estimator basis; counters also track conversion completion and the progressive web app installation funnel. The resulting dashboard exposed an estimate that was off by 83% and supports grouping real usage by conversion conditions, while sending only numeric metrics and not media, filenames, or URLs.
The Shared View That Makes Insurance AI Fast and Safe
Agentic AI transforms insurance claims processing, underwriting, and customer service by autonomously executing tasks, reducing a routine claims review from eight hours to eight minutes. However, adding AI agents to fragmented data silos of policy details, claims history, and buried email threads multiplies problems because agents work with incomplete views that degrade. Pairing AI with a unified data architecture that creates a single shared context layer is key; MongoDB Atlas powers this by storing all data in a flexible document model. This shared view delivers accelerated resolution through single-motion retrievals, seamless human oversight by basing decisions on the same record, and built-in compliance through an automatic action trail. Insurers who build on this foundation can safely scale AI agents across claims, underwriting, and customer service while maintaining the oversight regulators and customers expect.
Rerouting the Stream: How Lyft Moved to the Apache Flink Operator
Lyft migrated its fleet of hundreds of critical streaming jobs from an internally developed Flink Kubernetes operator to the open-source Apache Flink Kubernetes operator. The legacy system imposed maintenance burden, feature gaps, outdated dependencies, fragile savepoint handling, and a rigid memory model that complicated varied workloads, including Apache Beam applications. Lyft used an incremental migration designed to preserve deployment patterns and team workflows while addressing CRD translation, state-machine behavior, and operational compatibility. The new platform added explicit last-state upgrades, autoscaling, automatic memory tuning, and safer upgrade mechanisms, though autotuning required trade-offs with in-place autoscaling and node capacity planning. After the multi-quarter transition, maintenance burden was minimized, the autoscaler right-sized a fleet previously overprovisioned by a few million dollars annually, and Lyft could contribute fixes upstream instead of maintaining its own operator.
Testing cookie behavior across hundreds of web surfaces with our in-house auditor
Maintaining cookie consent behavior across more than 200 web surfaces is difficult because URLs, integrations, experiments, languages, and page settings change over time. Dropbox built an in-house cookie auditor that uses Playwright and fresh isolated browser sessions to test three experiences—a standard US visitor, an EU visitor, and a visitor sending a Global Privacy Control signal. Each test records cookies before interaction, finds consent controls without relying on visible button text, declines non-essential cookies, reloads the page, and verifies that the resulting cookies and saved preference match expected behavior. A companion URL detector narrows the audit set from traffic records, while configurable classifications, exceptions, weekly reports, and historical results support review, trend detection, and follow-up by Privacy and Engineering teams.
Stop wasting GPU allocation in Kubernetes with GPU-pruner
GPU-pruner addresses idle GPU allocation in Kubernetes by identifying workloads whose hardware activity remains below a configured threshold, rather than relying on browser sessions, network traffic, or pod lifetime. It queries Prometheus for NVIDIA Data Center GPU Manager metrics, including engine activity and GPU utilization, and treats peak performance below 1% throughout a default 35-minute observation window as idle. After tracing an idle pod through ownerReferences or KServe labels, it scales the parent resource to zero, preserving workload metadata for later restoration. Slack notifications, grace periods, lookback checks, and dry-run mode provide safeguards, while Prometheus label handling must preserve the workload namespace and pod identity in DCGM telemetry.
How Indra unified EV charging data on Databricks
Indra Renewable Technologies consolidated a fragmented EV-charging data estate spanning Cosmos DB, Synapse, Azure Data Lake Storage, Azure Functions, Power BI, and other services onto Databricks. The migration began with a parallel Synapse prototype and expanded to Delta tables, a consolidated fleet pipeline, and greater use of Databricks for operational workloads. A medallion architecture stores raw fleet data in bronze, transforms it with PySpark in silver, and publishes gold tables for reporting and client delivery; one pipeline now serves three clients and improved performance by 60% to 70%. Unity Catalog, serverless SQL warehouses, AI/BI dashboards, and Genie Agents support governed self-service analytics, while reported results include 80% to 90% business cost savings, 90% storage savings, and query latency falling from 24.4 seconds to 3.5 seconds.
Self-service backup for VMs and containers on OpenShift: No cluster-admin required
In multi-tenant Red Hat OpenShift clusters, Velero Backup and Restore resources live in the OADP installation namespace, leaving non-admin application teams dependent on platform staff. OADP self-service addresses this with namespace-scoped NonAdminBackup and NonAdminRestore resources that proxy Velero resources, while a controller validates requests against DataProtectionApplication policies and synchronizes phase, progress, and queue status back to the tenant namespace. Administrators can enforce retention, snapshot and file-system backup settings, resource inclusion or exclusion, and approval for tenant-provided backup storage; cluster-scoped resources such as SCCs, ClusterRoles, CRDs, and KubeVirt cluster preferences are automatically excluded. OADP 1.6 supports the same workflow for containers, stateful applications, and OpenShift Virtualization VMs, but self-service has no native schedules, cross-namespace restores, or short-lived credentials for tenant-provided storage locations.
Building hermetic notebook images for Open Data Hub and Red Hat OpenShift AI
Open Data Hub and Red Hat OpenShift AI adopted hermetic container builds to eliminate network failures, satisfy Conforma compliance, and ensure reproducible notebook environments. Instead of pulling packages during image creation, dependencies are pinned in lockfiles, prefetched into a local cache using tools like Cachi2, and installed with the network disabled. The workflow relies on a unified Containerfile capable of running locally, within GitHub Actions, and on Konflux via Tekton tasks. To pass audit checks and avoid generic tarball downloads, the team packaged awkward binaries like ripgrep and Pandoc into multi-arch Python wheels while patching Node postinstall scripts that fetch files at install time. Upstream CentOS and downstream Red Hat Enterprise Linux dependencies are maintained in distinct lock trees to prevent subtle packaging mismatches.
Data Mesh at Grab (Part III): Operationalizing data reliability with automated DPIs
Grab’s data mesh uses the Signals Marketplace to publish, discover, and reuse trusted data products, while certification establishes ownership, documentation, lineage, and quality controls but does not prevent runtime failures. Data Production Issues (DPIs) turn data-quality signals into an operational workflow that begins when Kinabalu detects unhealthy contract tests and closes when the underlying condition recovers. Triage promotes meaningful contract breaches, deduplicates repeated signals, groups related issues, and records affected assets, intervals, contracts, tests, and dependencies. For diagnosis, the Data Health API exposes a portable taxonomy of UPSTREAM_ERROR, PLATFORM_ERROR, JOB_ERROR, and DATA_ERROR, allowing ownership to follow the likely root cause while platforms retain detailed internal diagnostics. Combined with Hugo’s diagnosis and remediation patterns, the approach is presented as reducing resolution time and feeding operational reliability back into certification.
How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache
Cloudflare's Big Pineapple platform, which powers 1.1.1.1 and other DNS services, stores over 250 billion cache entries simultaneously. To decrease fleet-wide memory consumption, engineers implemented five changes to in-memory cache data structures. Key modifications included swapping growable types like Vec and String for Box slices to eliminate capacity overhead, combining distinct record sections into a single list indexed by offsets, omitting redundant owner domain names, and serializing record bytes into contiguous buffers. In benchmarks, these optimizations decreased per-entry memory footprints by 56% while improving cache lookup latency by 19% and insert throughput by 43%. Across production instances, the rollout reduced working-set memory by approximately 100 terabytes.
The Cheapest Token You Never Send: Agentic AI Search on MongoDB + Voyage AI
The post reframes cost, latency, and answer quality in agentic AI search as a single retrieval precision problem. Its pipeline embeds with flexible Voyage AI dimensionality and automatic quantization in MongoDB index, with binary reducing RAM by roughly 24x. Hybrid search via $rankFusion combines vector and full-text search with metadata filters in one server-side query, eliminating separate systems. A Voyage reranker then reorders the broad candidate set by true relevance, allowing the LLM to receive only the top few high-signal chunks and drastically shrinking input context. The agentic loop over consolidated Atlas where data, vectors, embeddings, and reranking coexist minimizes network hops per iteration, and an illustrative example shows an 80% reduction in retrieved-context input tokens.
A universal interface: How QuintoAndar made ClickHouse plug-and-play with managed Postgres
QuintoAndar’s customer data platform was split between a Postgres-based hot layer for low-latency lookups and a Databricks-based cold layer for analytical processing, creating duplicate code and expensive MERGE INTO operations. The team consolidated the pipeline by having its event gateway write directly to ClickHouse Cloud, which also serves API requests and analytical queries while remaining accessible to Trino, Spark, and Superset. Hightouch’s Customer Studio did not support ClickHouse, so QuintoAndar used ClickHouse Managed Postgres and the pg_clickhouse extension to expose ClickHouse tables as Postgres tables. Queries are pushed down to ClickHouse, while Hightouch’s planner and audit tables remain in NVMe-backed Postgres, allowing the existing activation workflow to continue with negligible round-trip impact for that use case.
From Preferences to Principles: Rubric-Based Alignment for Grounded Knowledge Answers
Designing reward signals for open-domain question answering remains challenging because high-quality answers must satisfy multiple distinct criteria that holistic scalar objectives fail to capture. To address this limitation, researchers developed a rubric-based reward framework that produces query-specific rubrics grounded in retrieved evidence and decomposed into specific quality dimensions. This structured reward setup provides fine-grained supervision during the post-training phase. Across the evaluation axes of composition, grounding, and instruction-following, the multi-dimensional grounded rubric method outperforms instruction-tuned baselines by 6.5% and flat rubric variants by 4%. Grounding rubrics in retrieved evidence bolsters factual support, while dimension decomposition enhances organization, coherence, and requirement adherence.
Gallup scales real-time coaching for thousands with Amazon Bedrock
Gallup needed to centralize decades of proprietary workplace performance research to provide proactive, real-time coaching for organizational leaders directly within their daily workflows. To solve this challenge, Gallup developed Gallup AI inside its Gallup Access platform using Amazon Bedrock and Anthropic Claude models. The multi-tenant serverless architecture leverages Amazon Bedrock Knowledge Bases and Amazon Kendra for retrieval-augmented generation, while AWS Lambda with FastAPI powers real-time streaming responses. Amazon ElastiCache Serverless caches recent conversation history for sub-millisecond session retrieval, Amazon RDS for MySQL provides durable transactional storage, and Amazon Bedrock Guardrails enforce content safety. Since its June 2024 launch, the system has achieved sub-second time-to-first-byte streaming latency while prompt volume grew approximately sevenfold across thousands of organizations.
Closing the AI agent trust gap with graduated autonomy
Operating AI agents under binary full-access or read-only permissions creates an operational trust gap due to unpredictable model behavior across updates. The graduated autonomy architectural pattern closes this gap by dynamically granting or revoking tool permissions through sustained reliability scoring. A scoring engine computes a composite score across five weighted dimensions over a rolling 50-action window, mapping agents across four tiers from probation to full autonomy. In-process pre-execution filters check for prompt injection and sensitive targets, while Amazon Bedrock AgentCore Gateway and Cedar policies strictly enforce deny-by-default access at the infrastructure level. Post-execution state capture in Amazon DynamoDB supports reversibility, and AWS CodePipeline delivery gates block releases failing adversarial evaluation benchmarks.
amazon.science
When LLM judges agree, should we believe them?
Multijudge panels in LLM-as-a-judge evaluations often suffer from correlated errors when models share prompt templates, training lineages, or evaluation rubrics. To account for these interdependencies without human reference labels, a method presented at ICML uses Ising models to simultaneously infer judge reliability and pairwise correlation. The unsupervised algorithm iteratively updates the soft probability of each item's true latent label while estimating both individual judge skill and pairwise dependence structures. Evaluated across retrieval relevance, toxicity classification, and summarization benchmarks with ten zero-temperature judges, the approach achieved test accuracies of 0.912, 0.792, and 0.806. These dependence-aware aggregation scores consistently outperformed weighted majority voting baselines by 9% to 14% on standard metrics across all three tasks.
Scaling Conditional Learned Retrieval for Pinterest Home Feed
Pinterest home feed candidate generation relies on large-scale retrieval, but standard two-tower models produce single embeddings that struggle to capture diverse concurrent user intentions. To solve this, Pinterest expanded Conditional Learned Retrieval (CLR) to generate condition-aware embeddings conditioned on explicit interest, Pin, and Board contexts. The team evolved model representations by appending condition tokens to user sequences in transformers, integrating the PinFM foundation model, and introducing hierarchical semantic IDs to improve cold-start coverage. On the serving side, engineers consolidated duplicate feature requests and shifted inference to GPU instances using NVEmbed under a one-query-to-many-documents paradigm. These infrastructure and modeling changes unlocked new condition types, achieved seven-figure cost savings, and reduced p90 model retrieval latency from 80ms down to 12ms.
Building a data plane from scratch: Stripe’s own high-performance distributed proxy
Stripe replaced Envoy in its service mesh after reliability, connection-cardinality, rollout-risk, and customization problems became increasingly difficult at its scale. Mesh-proxy, a Go proxy, implements HTTP/1.1 and HTTP/2, connects directly to Stripe’s service registry, and supports retries, hedging, rate limiting, and dynamic priority-based routing. Unlike Envoy’s worker-per-core model, it uses one read and one write goroutine per connection and multiplexes requests over a shared connection, reducing connection counts from one per worker to one per host. Load tests and deployment results showed about half the CPU under high load and roughly 50% lower latency, equivalent to around 3,000 days of cumulative inter-service latency saved daily. Direct registry integration also reduced unnecessary xDS updates, while custom Ruby balancing added 15–20% headroom before bursts overwhelm machines; Stripe is now pursuing anomaly detection and smart routing.
Migrating to Auth0 for Startups from Another Auth Solution
Evenpay develops compensation transparency software to meet the EU Pay Transparency Directive, requiring strict safeguards for sensitive employee salary data. The startup initially selected a lightweight B2B authentication platform for quick onboarding, but soon encountered unmet data residency and compliance demands from Finnish enterprise clients. To resolve these limitations, Evenpay migrated all customers to Auth0 over a few weeks using documentation and AI assistance. Adopting Auth0 allowed the team to fulfill regional EU data controls while reducing managed Single Sign-On setup times from weeks to as little as one day through self-service options. Furthermore, relying on an established authentication brand provided instant credibility with enterprise security officers without requiring custom authentication development.
PROOF-Gen: From Optimized Data to Better Distillation
Standard distillation pipelines for tool-calling agents use generate-and-filter supervised fine-tuning on teacher trajectories, discarding failed trials and missing training signals on hard scenarios. On τ 2-bench, 57% of teacher trials fail, with two-thirds being near-misses undone by a single error. PROOF-Gen (Per-scenario Reflective Optimization to Overcome Failed Generation) addresses this loss by employing a reflector that reviews execution traces and evaluation feedback to craft corrective guidance for the teacher. Once passing trajectories are generated, this scaffold guidance is stripped so students train on clean demonstrations. Per-scenario prompt optimization recovers 93% of failed scenarios on τ 2-bench, boosting Qwen3-4B-Instruct-2507 Passˆ1 from 0.132 to 0.529 and lifting deployed on-device model performance across all locales.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents540
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.