# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [Migrating Counter Service storage: Design choices and learnings](https://yomu.fyi/post/migrating-counter-service-storage-design-choices-and-learnings.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jia Long Loh
- Published: Jul 3, 2026

Grab migrated its Counter Service storage backend from a legacy wide-column database to Aerospike to support real-time anti-fraud windowed aggregations across tens of thousands of queries per second. To decouple storage from the Rust reader service, engineers introduced a storage facade using enum dispatch, avoiding the per-query heap allocations of boxed trait objects. The reader used configuration-driven operating modes to support shadow reads and deterministic traffic splitting without requiring code updates. On the write path, the schema was redesigned to collapse multiple bucket records into a single sorted map per counter, using atomic server-side operations to increment counters and prune expired entries. This data model redesign reduced total record counts and in-memory primary index usage by over an order of magnitude while enabling a zero-downtime transition.


### [Meta’s AI Storage Blueprint at Scale](https://yomu.fyi/post/meta-s-ai-storage-blueprint-at-scale.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Sidharth Bajaj, Venkatraghavan Srinivasan
- Published: Jul 1, 2026

Meta redesigned its BLOB-storage architecture on top of the foundational Tectonic block layer to eliminate GPU stalls and accelerate AI training workflows. The legacy storage system suffered from multi-layered metadata lookups, cross-region latency, and dataplane proxy bottlenecks that conflicted with the millisecond access requirements of flash-based AI clusters. To resolve these issues, the team collapsed disparate metadata into a single flat schema backed by ZippyDB for O(1) path lookups and introduced a fat client SDK capable of streaming data directly from storage servers. The updated stack also leverages spare GPU host memory as a distributed data cache alongside distributed read-plan caches to handle severe traffic spikes during checkpointing and model loading. These protocol and architectural changes deliver predictable latencies, reduce power consumption, and maintain high throughput during distributed training.


### [10 Years of Meta’s Commitment to Python](https://yomu.fyi/post/10-years-of-meta-s-commitment-to-python.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Published: Jun 30, 2026

Meta has marked ten consecutive years as a sponsor of the Python Software Foundation (PSF) to ensure the language remains robust and sustainable. Python represents the most widely used programming language across Meta's infrastructure, powering applications like Instagram and Threads as well as internal AI research. Beyond financial backing, Meta engineers actively contribute core language features, author Python Enhancement Proposals (PEPs), and build open-source tools such as the Pyrefly type checker. Company sponsorship directly funds the PSF's Developer-in-Residence program, core infrastructure, and security enhancements for the Python Package Index (PyPI). Continued support of the PSF functions as a strategic investment in maintaining the stability of the broader open-source ecosystem.


### [GenPage: Towards End-to-End Generative Homepage Construction at Netflix](https://yomu.fyi/post/genpage-towards-end-to-end-generative-homepage-construction-at-netflix.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jun 29, 2026

Netflix traditionally constructs its structured two-dimensional homepage through a complex, multi-stage recommender pipeline that separates candidate generation and ranking across rows and entities. To simplify this architecture and optimize directly for whole-page user satisfaction, Netflix developed GenPage, an end-to-end generative transformer model that autoregressively builds the entire homepage from raw tokenized context. The system relies on a domain-specific tokenizer to compress engagement history, context injection for cold start, hybrid row decoding to minimize decoding steps, and reinforcement learning post-training. In online A/B testing against the production baseline, GenPage delivered statistically significant gains in core user engagement metrics while cutting end-to-end serving latency by 20 percent. Offline evaluations further showed that enriching context prompts improved recommendation quality more effectively than increasing model capacity in the current operating regime.


### [How we used DSPy to turn AI evaluations into better responses in Dash chat](https://yomu.fyi/post/how-we-used-dspy-to-turn-ai-evaluations-into-better-responses-in-dash.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon
- Published: Jun 25, 2026

Dropbox improved its Dash chat agent by establishing an automated optimization loop powered by DSPy and LLM-as-judge evaluations. Engineers first calibrated their LLM judges against human-annotated interaction traces, then used those judges to systematically optimize the agent's system prompts via offline counterfactual replay. This automated workflow reduced incomplete responses by 26% while decreasing overall token consumption.


### [Toward More Controllable AI Video Editing: An Early Research Exploration at Netflix](https://yomu.fyi/post/toward-more-controllable-ai-video-editing-an-early-research-exploratio.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jun 23, 2026

Generative video editing tools often regenerate entire video clips, which inadvertently modifies untouched scene details or violates physical continuity during object deletion. To provide artists with precise control, two research prototypes were developed: Vera and VOID. Vera uses a layered video diffusion framework with a Mixture-of-Transformers architecture to jointly generate separate edit and alpha matte layers, compositing them with original footage while leaving unchanged pixels untouched. VOID addresses object deletion by conditioning on interaction-aware quadmasks and using a second-pass appearance refiner to reconstruct scenes with plausible physical continuity. In a user study evaluated across 75 real-world scenarios, reviewers selected VOID 64.8% of the time over six baselines.


### [How Netflix Simplified Batch Compute with Kueue](https://yomu.fyi/post/how-netflix-simplified-batch-compute-with-kueue.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jun 22, 2026

Netflix transitioned its managed batch compute infrastructure from a homegrown solution called Compute Managed Batch to Kueue on its Titus container platform. CMB previously relied on custom scheduling and admission-only fair sharing without preemption, making feature development cumbersome as the Kubernetes ecosystem evolved. To modernize the platform transparently, Netflix mapped internal tenants to Cohorts and leaf tenants to ClusterQueues and LocalQueues while routing jobs through a custom Kueue router. Kueue operates alongside existing Titus scheduling profiles rather than replacing the kube-scheduler, preserving cluster placement efficiency. The migration was completed in four weeks across millions of batch workloads, significantly increasing average resource utilization through preemption-based fair sharing.


### [Scaling out Distroless adoption With AI](https://yomu.fyi/post/scaling-out-distroless-adoption-with-ai.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jia Yee Chong
- Published: Jun 22, 2026

Grab is transitioning its microservices to Distroless base images to eliminate unnecessary binaries and reduce vulnerability risks, but the migration risks runtime failures from missing shared objects and system utilities. To safely validate container execution in continuous integration without staging dependencies, the team relied on medium tests that run containerized services alongside internal dependencies managed by Testcontainers. Because hundreds of services lacked this test harness, Grab implemented an agentic workflow using Claude Code and Model Context Protocol integrations to inspect repositories, generate test boilerplate, and resolve configuration errors. Once test baselines are established, an automated patch-test-compare pipeline updates Dockerfiles, constructs multi-stage builds for necessary dynamic libraries, and creates draft merge requests for human approval.


### [Palana (Part 2): Architecting isolation, identity, and auditability for AI agents](https://yomu.fyi/post/palana-part-2-architecting-isolation-identity-and-auditability-for-ai.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Kevin Littlejohn
- Published: Jun 21, 2026

Grab's Palana platform provisions isolated, Kubernetes-native runtime environments for autonomous AI agents using dedicated per-agent namespaces and role-based access controls. The architecture separates network enforcement across layers, applying Layer 3 and Layer 4 containment with Cilium and NetworkPolicy alongside Layer 7 application filtering evaluated by Open Policy Agent. Agent interactions with large language models route through a LiteLLM proxy wrapper that retrieves credentials from HashiCorp Vault based on Kubernetes pod context rather than client headers. Secrets management is divided between directly readable agent paths and proxy-only placeholder paths that prevent raw tokens from residing in runtime filesystems. An automated reaper monitors multi-source activity signals to shut down idle compute resources while preserving persistent storage and configuration state.


### [The Data Canary: How Netflix Validates Catalog Metadata](https://yomu.fyi/post/the-data-canary-how-netflix-validates-catalog-metadata.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jun 19, 2026

A manual mitigation action during an incident corrupted a data feed for a subset of titles, causing playback issues and catalog service failures that existing code canary systems failed to catch. To protect streaming reliability, Netflix built an automated data canary system that validates transformed catalog metadata prior to publication. The architecture utilizes a dedicated orchestrator alongside permanent baseline and canary service clusters to coordinate validation using real production traffic. By leveraging custom chaos experiment thresholds, sticky session affinity, and Starts Per Second playback metrics, the system detects regressions in under ten minutes and blocks publication automatically. Controlled failure injection experiments routing approximately 0.2% of global traffic confirmed that issues could be identified in 2.5 to 4 minutes.


### [Palana (Part 1): Why Grab built a secure platform for autonomous AI Agents](https://yomu.fyi/post/palana-part-1-why-grab-built-a-secure-platform-for-autonomous-ai-agent.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Kevin Littlejohn
- Published: Jun 19, 2026

Autonomous AI agents introduce significant operational and security risks when granted network access, persistent state, and credentials. To address these concerns without impeding developer productivity, Grab created Palana, an in-house Kubernetes-native execution substrate. The platform isolates each agent workload within its own namespace, pairing it with dedicated storage, network policies, and role-based access control. Network egress is funneled through an Envoy and Open Policy Agent proxy layer that audits requests and injects credentials from HashiCorp Vault using placeholder tokens, keeping raw secrets outside the agent runtime. This design allows Grab to securely host hundreds of long-running workflows, remote coding environments, and automation bots.


### [How Dropbox uses MCP and Dash to close the design-to-code security gap](https://yomu.fyi/post/how-dropbox-uses-mcp-and-dash-to-close-the-design-to-code-security-gap.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra
- Published: Jun 12, 2026

Dropbox developed a system using the Model Context Protocol (MCP) and Dash's semantic search to bridge the gap between security threat models and code implementation. By retrieving original security documents during pull requests, an LLM agent automatically evaluates whether the proposed code adheres to previously agreed-upon security requirements. This approach surfaces design regressions and missing controls that traditional static analysis tools miss.


### [Agentic Testing: Where Agents Fit in the E2E Testing Stack](https://yomu.fyi/post/agentic-testing-where-agents-fit-in-the-e2e-testing-stack.md)
- Company: [Slack](https://yomu.fyi/company/slack.md)
- Author: Sergii Gorbachov
- Published: Jun 11, 2026

Traditional end-to-end tests validate rigid user journeys, whereas agentic tests verify whether broad goals can be achieved by adapting actions dynamically. To evaluate agentic testing tradeoffs, researchers executed over 200 runs across Playwright Model Context Protocol (MCP), Playwright CLI, and agent-generated Playwright tests using Claude models. Playwright MCP demonstrated high reliability with failure rates of 0% on simple thread replies and approximately 12% on complex search discovery flows. Playwright CLI and generated code struggled more on complex workflows, exhibiting failure rates of approximately 20% and 48% respectively. Although generated tests were faster with average runtimes of roughly three minutes, agentic testing provides a distinct exploratory layer atop deterministic CI test suites.


### [Scaling beyond one: How Airbnb evolved its data architecture for a multi-product world](https://yomu.fyi/post/scaling-beyond-one-how-airbnb-evolved-its-data-architecture-for-a-mult.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Patrick Lam
- Published: Jun 9, 2026

To support its expansion from Homes into Experiences and Services, Airbnb evolved its offline data warehouse architecture to balance domain-specific needs with organizational consistency. Rather than enforcing a single global pattern, data teams adopted a framework of foundational principles that paired separate data models for unique product features with monolithic models for cross-cutting domains like payments and messaging.


### [Sitar-agent: Building a reliable dynamic configuration sidecar at scale](https://yomu.fyi/post/sitar-agent-building-a-reliable-dynamic-configuration-sidecar-at-scale.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Bo Teng
- Published: Jun 4, 2026

Airbnb modernized sitar-agent, a Kubernetes sidecar that delivers dynamic configuration updates to thousands of polyglot service instances without requiring redeployments. The architecture uses periodic AWS S3 snapshots to enable fast, decoupled pod startups and maintains local file-based storage for the main container to read configs with in-memory caching. Key design decisions include retaining an isolated sidecar over an in-process library and optimizing pull-based polling with server-side caching.


### [When history fails you, borrow from geography](https://yomu.fyi/post/when-history-fails-you-borrow-from-geography.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Harrison Katz
- Published: Jun 2, 2026

When historical travel data became uninformative during the asynchronous post-COVID recovery, Airbnb redesigned its demand forecasting to borrow signals across geographies rather than waiting for local data. Using a hierarchical Bayesian framework, the team propagated updated posterior estimates from early-recovering corridors as informative priors for structurally similar, later-recovering corridors. This allowed accurate corridor-level demand forecasting in near real time during periods of high disruption and uneven market reopening.


### [From decentralized Docs-as-Code to a centralized repository: Evolving Grab's documentation strategy](https://yomu.fyi/post/from-decentralized-docs-as-code-to-a-centralized-repository-evolving-g.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Karen Kue
- Published: May 29, 2026

Grab originally adopted a decentralized Docs-as-Code workflow to manage Markdown documentation alongside service code, but the model strained as engineering scaled. Dispersed repositories led to fragmented terminology, uncoordinated repository moves, uneven linting standards, and noisy search results. To resolve these issues, the organization transitioned to a centralized documentation repository with standardized templates, automated CI quality gates, and a unified search overlay combining fuzzy title search with Glean. Consolidating Markdown into a single repository also established a clean, structured knowledge base tailored for internal AI agents. The centralized setup fostered continuous maintenance, averaging 27 daily commits and an update merged roughly every 50 minutes.


### [Beyond code generation: rethinking engineering productivity in the age of AI agents](https://yomu.fyi/post/beyond-code-generation-rethinking-engineering-productivity-in-the-age.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura
- Published: May 28, 2026

Dropbox shares how widespread AI code generation shifts software development bottlenecks downstream into code review, CI infrastructure, and validation pipelines. To adapt, they built Nova, an internal coding agent platform that safely automates scoped tasks such as migrations and flaky test remediation. They also evolved their developer productivity framework to measure end-to-end customer impact and code quality rather than simple pull request throughput.


### [Slack AI: The Path to Multi-Cloud](https://yomu.fyi/post/slack-ai-the-path-to-multi-cloud.md)
- Company: [Slack](https://yomu.fyi/company/slack.md)
- Author: Shaurya Kethireddy
- Published: May 28, 2026

Slack evolved its Slack AI serving infrastructure across multiple phases to handle enterprise LLM workloads reliably and securely. The initial deployment on AWS SageMaker provided zero-knowledge escrow VPC isolation and FedRAMP compliance, but engineers faced scaling latency, GPU scarcity, and significant operational overhead. Slack then migrated live traffic to Amazon Bedrock to leverage managed Model Units and eliminate model release lag without customer-facing incidents. However, fixed Provisioned Throughput commitments and regional peak traffic variations created persistent underutilization challenges. Consequently, Slack expanded into a multi-cloud orchestration architecture that normalizes disparate provider APIs, integrates unified cross-cloud telemetry, and routes traffic dynamically around latency spikes and outages.


### [The Hugo evolution: Engineering Grab's unified, one-click data ingestion platform with Apache Flink](https://yomu.fyi/post/the-hugo-evolution-engineering-grab-s-unified-one-click-data-ingestion.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Shuguang Xiang
- Published: May 22, 2026

Grab's self-service data platform, Hugo, faced significant onboarding friction as streaming pipelines expanded across fragmented systems like Kafka Connect, custom Go applications, and Spark. Engineering teams struggled with cross-platform configuration translations and brittle, manual schema mappings that stretched onboarding over several days. To resolve these bottlenecks, Grab modernized the ingestion architecture by introducing a centralized automation layer powered by Apache Flink and Flink CDC. The updated platform dynamically retrieves Protobuf schemas from Confluent Schema Registry and ingests MySQL binlogs directly into queryable Hive tables without intermediate Kafka hops. This shift dropped pipeline onboarding times to roughly six minutes for Kafka and three minutes for MySQL CDC, driving more pipeline adoptions in one year than in the previous five.


[Newer posts](https://yomu.fyi/page/4.md) · [Older posts](https://yomu.fyi/page/6.md)