# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [Eval-driven development: Lessons from evaluating GenAI at scale](https://yomu.fyi/post/eval-driven-development-lessons-from-evaluating-genai-at-scale.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Rohit Girme
- Published: Jul 28, 2026

Airbnb outlines its framework for eval-driven development (EDD), treating GenAI evaluation as a continuous engineering discipline rather than an afterthought. The strategy combines programmatic checks, calibrated LLM-as-a-judge evaluators, and human review to detect subtle quality regressions. By inspecting intermediate agentic execution traces and real-world failure modes, teams ensure generative models remain reliable and aligned with product requirements.


### [Building Reliable Production AI with Durable Workflows](https://yomu.fyi/post/building-reliable-production-ai-with-durable-workflows.md)
- Company: [Salesforce](https://yomu.fyi/company/salesforce.md)
- Author: Scott Nyberg
- Published: Jul 27, 2026

AI prototypes rely on simple prompt-response interactions, but production AI systems handling thousands of records must manage distributed execution issues like worker crashes, rate limits, and transient failures. Rather than treating long-running operations as single large jobs or rebuilding state from logs, systems require durable workflows that preserve execution history across failures. In Agentforce Grid, execution is organized into a hierarchy where column runs divide into recoverable batches and rows composed of discrete activities. Each batch checkpoints after every successful row, retrying up to ten times with exponential backoff and resuming strictly at incomplete work to prevent duplicated model calls. Exposing progress across these hierarchical boundaries ensures that both users and engineers can distinguish completed, failing, and retryable units.


### [Agent platform (Part 1): How we help Grab build and run AI agents at scale](https://yomu.fyi/post/agent-platform-part-1-how-we-help-grab-build-and-run-ai-agents-at-scal.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Kendrick Tan
- Published: Jul 24, 2026

Grab scaled its internal AI tooling by transforming the infrastructure behind a single technical support bot into a standardized agent framework named LLM-Kit. The original Go-based bot revealed critical bottlenecks, including the lack of automated evaluation, painful provider migrations, fragmented observability, and excessive production scaffolding overhead. To address these frictions, engineers developed a FastAPI template pre-wired with LangGraph ReAct agents, remote Model Context Protocol servers, and built-in ROUGE, BLEU, and LLM-as-judge evaluation suites. The framework centralizes model access through an OpenAI-compatible GrabGPT Gateway and unifies observability across services, outbound calls, and agent steps using OpenTelemetry. This extracted scaffolding reduced day-one production setup time from two weeks to roughly an hour while supporting hundreds of production agents.


### [How AI Rebuilt Salesforce’s Decades-Old Localization Pipeline](https://yomu.fyi/post/how-ai-rebuilt-salesforce-s-decades-old-localization-pipeline.md)
- Company: [Salesforce](https://yomu.fyi/company/salesforce.md)
- Author: Scott Nyberg
- Published: Jul 23, 2026

Salesforce needed to deliver major releases across 34 languages as product localization volume surged by more than 35% under fixed budgets and unchanged delivery windows. To address these scaling constraints, the engineering team replaced their legacy translation workflows with an LLM-powered orchestration pipeline integrating prompt engineering, context engineering, and multi-stage validation. The architecture transforms existing style guides, glossaries, and terminology databases into structured context, routing interface strings through specialized prompt stages—up to 85 stages for some languages—to enforce product context, grammar, and branding rules. Combined with an automated validation loop spanning AI editing, AI validation, and human review, this system reduced localization costs by 50 to 90% while accelerating turnaround times across enterprise releases.


### [Personalizing Airbnb search by learning from the guest journey](https://yomu.fyi/post/personalizing-airbnb-search-by-learning-from-the-guest-journey.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Daochen Zha
- Published: Jul 21, 2026

Airbnb replaced hundreds of hand-crafted ranking features with a Transformer-based sequence model that captures both long-term booking history and short-term browsing behavior. To keep latency low and throughput high, sequence representations are generated via daily batch jobs and combined with real-time queries using a co-trained setwise ranker.


### [How our universal content processing platform Riviera evolved for AI and beyond](https://yomu.fyi/post/how-our-universal-content-processing-platform-riviera-evolved-for-ai-a.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash
- Published: Jul 20, 2026

Dropbox developed Riviera, a centralized content processing platform that evolved from an internal file preview generator into a shared infrastructure supporting over 300 file formats. By decomposing complex conversion tasks into modular, reusable transformations, the system now powers search indexing, media transcoding, and AI context extraction across Dropbox products.


### [In-House LLM Serving at Netflix](https://yomu.fyi/post/in-house-llm-serving-at-netflix.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jul 17, 2026

Netflix established an in-house serving platform to run large language model inference directly inside existing production environments alongside traditional machine learning models. Built upon NVIDIA Triton Inference Server and vLLM, the unified architecture handles member-scale routing, candidate generation, feature fetching, and model execution over gRPC and OpenAI-compatible HTTP endpoints. The team shifted their primary engine from TensorRT-LLM to vLLM to support custom architectures, simpler debugging, and non-trivial constraint logic. Addressing production obstacles required patching Triton's frontend for guided decoding, pinning dependent library versions, rewriting logits processors in C++ for vLLM V1, and handling state machine resets during engine preemptions. The resulting platform unifies deployment flows while preserving operational stability across evolving model schemas.


### [The cost of saying yes has changed](https://yomu.fyi/post/the-cost-of-saying-yes-has-changed.md)
- Company: [Github](https://yomu.fyi/company/github.md)
- Author: Dalia Abuadas
- Published: Jul 17, 2026

Generating initial code patches has become significantly cheaper with AI agents, shifting the primary expense of small feature requests from writing code to debating scope in meetings. Teams can use agent-generated patches as diagnostic probes rather than final deliverables, turning abstract scope debates into concrete artifacts that reveal true system touchpoints and risks. However, low generation costs do not translate to low ownership costs, as changes touching areas like authorization, compliance, or public contracts still demand significant human review. Constrained attempts allow engineers to price uncertainty quickly and shift scope discipline from pre-implementation speculation to evidence-based code review.


### [Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization](https://yomu.fyi/post/exploring-hierarchical-interest-representation-for-meta-ads-deep-funne.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Yuhui Ouyang, Di Wang, Sreedal Menon, Jie Tian
- Published: Jul 15, 2026

Meta developed Hierarchical Interest Representation as an upstream representation layer to optimize deep funnel ad ranking across its platforms. The architecture organizes interactions among users, advertisers, and products into a heterogeneous, time-decayed graph and projects sparse raw edges into multi-level super-graphs of latent interest primitives. To encode these structures, a transformer-based hierarchical encoder incorporates multimodal product semantics processed via language models alongside topology-aware attention biases. Computation scales across billions of entities by pairing deep hash ID embeddings with FlexAttention to evaluate graph biases on the fly without materializing full matrices. Self-supervised cross-view distillation trains student representations against broader teacher graph views to yield universal embeddings and interest tokens.


### [From weeks to a day: how we made LLM evaluation fast enough to iterate on](https://yomu.fyi/post/from-weeks-to-a-day-how-we-made-llm-evaluation-fast-enough-to-iterate.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Baharak Saberidokht
- Published: Jul 14, 2026

Airbnb built a four-layer LLM infrastructure framework to reduce iteration and evaluation turnaround from weeks to a single day. By using per-sample caching for generated references and judge scores, they established a deterministic evaluation foundation that separates genuine model drift from measurement noise. This setup enables rapid, bounded model hotfixes using micro LoRA adapters and comprehensive end-to-end validation across component boundaries.


### [Shipyard: How We Built Slack’s Next-Generation EC2 Platform](https://yomu.fyi/post/shipyard-how-we-built-slack-s-next-generation-ec2-platform.md)
- Company: [Slack](https://yomu.fyi/company/slack.md)
- Author: Archie Gunasekara
- Published: Jul 14, 2026

Slack developed Shipyard to modernize its EC2 infrastructure by replacing continuously updated, mutable instances with immutable deployable artifacts. The platform adopts a layered image architecture starting with a standardized base image called slack-zero, which is built using AWS Image Builder and consumed by downstream service-specific machine images. Rather than running periodic Chef jobs in the background, configuration management is restricted to image baking and initial instance provisioning phases. Fleet management is handled through the Peekaboo inventory system, metric-driven progressive rollouts via Gondola, and automated instance rotation managed by a lifecycle component known as the Reaper. Developers validate cookbook updates prior to merging using Ship Quick, which provisions real infrastructure across dedicated worker fleets to verify provisioning workflows.


### [Building Service Topology at Scale: Architecture, Challenges, and Lessons Learned](https://yomu.fyi/post/building-service-topology-at-scale-architecture-challenges-and-lessons.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jul 13, 2026

Engineers at Netflix required a unified, real-time view of service dependencies to navigate distributed architecture and improve incident troubleshooting. Traditional batch systems introduced stale data, so the team created a streaming-first platform backed by reactive streams and backpressure handling to ingest flow records from multi-region Kafka streams and Server-Sent Events without data loss. The architecture partitions data into physically separate graph and columnar storage layers covering eBPF network flows, IPC metrics, and distributed traces. Network flow ingestion relies on a three-stage distributed aggregation pipeline using consistent hashing to resolve network intermediaries into logical application connections. The resulting production system serves time-travel and topology queries with sub-second latency while continuously updating dependency views.


### [Modernizing the Meta Ads Service With an Open-Source Kernel Scheduler](https://yomu.fyi/post/modernizing-the-meta-ads-service-with-an-open-source-kernel-scheduler.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Published: Jul 13, 2026

Meta's ad serving fleet encountered latency regressions and technical debt after Linux kernel upgrades introduced the default EEVDF scheduler. To resolve these performance drops across billions of daily requests, the engineering team implemented sched\_ext, an open-source, BPF-based scheduling framework integrated into Linux kernel v6.12. The custom scheduling policy uses domain knowledge to soft-partition CPUs dynamically into separate pools for latency-critical request threads and background work, enhancing L3 cache locality. Packaged as a user-space binary, the scheduler allows policy iterations and rollouts via process restarts without kernel rebuilds. The deployment achieved a 28% reduction in p99 ads retrieval latency, a 1.1% increase in ranked ads, and 3.28 megawatts in fleet-wide power savings.


### [Better tools made Copilot code review worse. Here’s how we actually improved it.](https://yomu.fyi/post/better-tools-made-copilot-code-review-worse-here-s-how-we-actually-imp.md)
- Company: [Github](https://yomu.fyi/company/github.md)
- Author: Napalys Klicius
- Published: Jul 10, 2026

When migrating GitHub Copilot code review from custom navigation tools to shared Unix-style CLI utilities—grep, glob, and view—benchmarks revealed higher review costs and fewer caught issues. Rather than an issue with the underlying tools, trace analysis showed the agent used general-purpose coding assistant instructions that triggered expansive repository browsing loops. In response, the team rewrote the tool guidance to enforce a review-specific workflow anchored to pull request diffs. The updated instructions direct the agent to narrow candidate call sites and files using batched discovery before reading minimal line ranges with view. This workflow tuning reduced average review costs by approximately 20% in production while maintaining review quality.


### [Scaling Grab's Data Lake: Our journey to Apache Iceberg adoption](https://yomu.fyi/post/scaling-grab-s-data-lake-our-journey-to-apache-iceberg-adoption.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Rahul Penti
- Published: Jul 10, 2026

Grab transitioned its petabyte-scale data lake from Hive Parquet to Apache Iceberg to resolve metadata bottlenecks, small file fragmentation, and lack of native ACID support. Implementing Iceberg with Z-ordering reduced navigation query runtimes from 70 seconds to 6 seconds and cut daily S3 API costs by up to 95% on key operational datasets. To seamlessly manage concurrent table formats without forcing engineers to specify format prefixes, Grab developed and open-sourced UnifiedSparkCatalog. This custom catalog transparently detects table types via the Hive Metastore, routes queries to format-specific handlers, and retains backwards compatibility for legacy Hive operations.


### [Automating cross-repo documentation with GitHub Agentic Workflows](https://yomu.fyi/post/automating-cross-repo-documentation-with-github-agentic-workflows.md)
- Company: [Github](https://yomu.fyi/company/github.md)
- Author: David Pine
- Published: Jul 8, 2026

Maintaining documentation across separate repositories often leads to severe lag because technical writers must reverse-engineer shipped features weeks after release. To address this in the Aspire project, the team implemented an automated pipeline using GitHub Agentic Workflows to bridge the product and documentation repositories. When product pull requests merge, a bash step maps milestones to docs release branches before an LLM agent evaluates the diff, drafts documentation updates, and emits structured pull request intents. A dedicated safe-outputs handler materializes these drafts via a scoped GitHub App and assigns the original code reviewers to verify accuracy. Across 396 product pull requests, the system generated 82 documentation pull requests that all merged with a median turnaround time of 44.8 hours.


### [Migrating Counter Service storage: Design choices and learnings](https://yomu.fyi/post/migrating-counter-service-storage-design-choices-and-learnings.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jia Long Loh
- Published: Jul 3, 2026

Grab migrated its Counter Service storage backend from a legacy wide-column database to Aerospike to support real-time anti-fraud windowed aggregations across tens of thousands of queries per second. To decouple storage from the Rust reader service, engineers introduced a storage facade using enum dispatch, avoiding the per-query heap allocations of boxed trait objects. The reader used configuration-driven operating modes to support shadow reads and deterministic traffic splitting without requiring code updates. On the write path, the schema was redesigned to collapse multiple bucket records into a single sorted map per counter, using atomic server-side operations to increment counters and prune expired entries. This data model redesign reduced total record counts and in-memory primary index usage by over an order of magnitude while enabling a zero-downtime transition.


### [Meta’s AI Storage Blueprint at Scale](https://yomu.fyi/post/meta-s-ai-storage-blueprint-at-scale.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Sidharth Bajaj, Venkatraghavan Srinivasan
- Published: Jul 1, 2026

Meta redesigned its BLOB-storage architecture on top of the foundational Tectonic block layer to eliminate GPU stalls and accelerate AI training workflows. The legacy storage system suffered from multi-layered metadata lookups, cross-region latency, and dataplane proxy bottlenecks that conflicted with the millisecond access requirements of flash-based AI clusters. To resolve these issues, the team collapsed disparate metadata into a single flat schema backed by ZippyDB for O(1) path lookups and introduced a fat client SDK capable of streaming data directly from storage servers. The updated stack also leverages spare GPU host memory as a distributed data cache alongside distributed read-plan caches to handle severe traffic spikes during checkpointing and model loading. These protocol and architectural changes deliver predictable latencies, reduce power consumption, and maintain high throughput during distributed training.


### [10 Years of Meta’s Commitment to Python](https://yomu.fyi/post/10-years-of-meta-s-commitment-to-python.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Published: Jun 30, 2026

Meta has marked ten consecutive years as a sponsor of the Python Software Foundation (PSF) to ensure the language remains robust and sustainable. Python represents the most widely used programming language across Meta's infrastructure, powering applications like Instagram and Threads as well as internal AI research. Beyond financial backing, Meta engineers actively contribute core language features, author Python Enhancement Proposals (PEPs), and build open-source tools such as the Pyrefly type checker. Company sponsorship directly funds the PSF's Developer-in-Residence program, core infrastructure, and security enhancements for the Python Package Index (PyPI). Continued support of the PSF functions as a strategic investment in maintaining the stability of the broader open-source ecosystem.


### [GenPage: Towards End-to-End Generative Homepage Construction at Netflix](https://yomu.fyi/post/genpage-towards-end-to-end-generative-homepage-construction-at-netflix.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jun 29, 2026

Netflix traditionally constructs its structured two-dimensional homepage through a complex, multi-stage recommender pipeline that separates candidate generation and ranking across rows and entities. To simplify this architecture and optimize directly for whole-page user satisfaction, Netflix developed GenPage, an end-to-end generative transformer model that autoregressively builds the entire homepage from raw tokenized context. The system relies on a domain-specific tokenizer to compress engagement history, context injection for cold start, hybrid row decoding to minimize decoding steps, and reinforcement learning post-training. In online A/B testing against the production baseline, GenPage delivered statistically significant gains in core user engagement metrics while cutting end-to-end serving latency by 20 percent. Offline evaluations further showed that enriching context prompts improved recommendation quality more effectively than increasing model capacity in the current operating regime.


[Newer posts](https://yomu.fyi/page/3.md) · [Older posts](https://yomu.fyi/page/5.md)