# Architecture
> 173 posts about Architecture, summarised, each linking to the original.

## Articles

### [How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC…](https://yomu.fyi/post/how-and-why-netflix-built-a-real-time-distributed-graph-part-3-queryin.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Aug 7, 2026

Netflix designed a query execution serving layer for its Real-Time Distributed Graph to power sub-100ms responses across diverse graph traversal workloads spanning billions of nodes and edges. To prevent compounding network delays in multi-hop queries, the engine uses a breadth-first traversal model that batches entity lookups across entire frontier levels rather than tracing paths depth-first. The entire serving architecture relies on asynchronous composition across small thread pools of 16 to 24 threads, ensuring no thread blocks while waiting on remote storage or enrichment I/O. Selective caching via EVCache targets stable properties with volatility-matched TTLs, producing 70 to 80 percent cache hit rates and reducing backend storage calls by three to four times. A layered filtering hierarchy pushes depth and edge limits directly to the storage tier, eliminating bespoke code changes while isolating system resources against excessive fan-out.


### [Unveiling good and bad behaviors on the Agentic Internet](https://yomu.fyi/post/unveiling-good-and-bad-behaviors-on-the-agentic-internet.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Jin-Hee Lee
- Published: Aug 7, 2026

Web traffic increasingly shifts between human interaction and automated agents, rendering point-in-time security checks ineffective against sophisticated automation. Cloudflare addresses this shift through continuous behavioral analysis designed to evaluate trust across entire user sessions rather than relying on static point-in-time hurdles. Its CDN-injected client-side system, Precursor, tracks continuous behavioral indicators like cursor movements to catch mid-session automated takeovers. To counter rapid bot adaptation, Cloudflare is introducing self-adjusting machine learning models under Adaptive Intelligence and non-deterministic defenses such as the AI Labyrinth. These tools allow site owners to poison scraper training data, trap crawlers in synthetic webs, or queue legitimate agents while dynamically penalizing abusive traffic.


### [Introducing Radar Researcher: An AI tool for exploring Internet data in plain language](https://yomu.fyi/post/introducing-radar-researcher-an-ai-tool-for-exploring-internet-data-in.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Lai Yi Ohlsen
- Published: Aug 7, 2026

Cloudflare launched the beta of Radar Researcher, an AI tool designed to let users explore global Internet traffic data using plain-language queries. Built entirely on Cloudflare's developer platform, the system uses the Cloudflare Agents SDK within a Cloudflare Worker, maintaining conversation state and SQLite databases inside Durable Objects. The inference layer relies on Workers AI running models like Kimi K2.7 through an ordered fallback chain across three model families to prevent outages, routed via AI Gateway for logging, caching, and guardrails. Radar's frontend renders retrieved data using its existing native interactive charts, while smaller dedicated models generate conversation titles and follow-up suggestions asynchronously. Additionally, Radar introduced support for the WebMCP standard, exposing imperative and declarative tools to allow external browser agents to navigate and interact with the platform directly.


### [Unifying Workers AI and AI Gateway into a single AI control plane](https://yomu.fyi/post/unifying-workers-ai-and-ai-gateway-into-a-single-ai-control-plane.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Michelle Chen
- Published: Aug 7, 2026

Workers AI and AI Gateway are converging into a unified AI control plane to simplify model access, billing, and observability. Developers can now access both services through a single entrypoint using unified Workers bindings and the /ai/ REST endpoint. Specifying a default gateway automatically instruments requests with payload logging, token tracking, and cost attribution without manual dashboard setup. Cloudflare has also rolled out unified prepaid billing, allowing teams to spend AI Gateway credits on Workers AI models while unlocking higher rate limits. Planned future features include model-first routing for automated provider failover and smart routing powered by task classifiers on Workers AI.


### [The next generation of MCP](https://yomu.fyi/post/the-next-generation-of-mcp.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Matt Carey
- Published: Aug 6, 2026

Model Context Protocol previously required stateful connections between clients and remote servers, creating operational complexity around sticky sessions, message replay, and open streams. The newly released MCP 2026-07-28 specification transitions MCP to a fully stateless protocol alongside updated SDKs across TypeScript, Python, Go, and C#. The protocol eliminates mandatory handshakes and session identifiers from core request paths while enabling servers to run on request-scoped infrastructure like Cloudflare Workers. Interactive elicitation is restructured through Multi Round-Trip Requests, which allow servers to return input requirements without sustaining persistent connections. Standardized headers for method names and improved OAuth authorization mechanisms further enable standard web infrastructure to inspect and manage MCP traffic efficiently.


### [Building an open Agentic Internet: readable, discoverable, callable, and payable](https://yomu.fyi/post/building-an-open-agentic-internet-readable-discoverable-callable-and-p.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Jack Galilee
- Published: Aug 6, 2026

The modern web produces billions of bot requests that repeatedly fetch unchanged pages, highlighting an architecture fundamentally misaligned with automated visitors acting on behalf of paying users. Cloudflare outlines an open Agentic Internet structured around four foundational capabilities: readability, discoverability, callability, and payability. Identity and access rely on open specifications such as Web Bot Auth and Private Access Control Tokens, while Markdown for Agents and the Kitesurf browser reduce token overhead and context window bloat. For execution and monetization, WebMCP exposes structured tools directly through the browser, and protocols like x402 paired with Monetization Gateway enable per-fetch transactions. These open standards allow domain owners to define tool contracts, verify bot identities, and monetize agent traffic without relying on legacy advertising models.


### [From User Sequences to Scaling Laws: A Multi-Stage Architecture for Meta’s Ads Ranking](https://yomu.fyi/post/from-user-sequences-to-scaling-laws-a-multi-stage-architecture-for-met.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Steven De Gryze, Parshva Doshi, Sean O'Byrne, Arnold Overwijk, Dinesh Ramasamy, Lee Xiong
- Published: Aug 5, 2026

Scaling sequence models for real-time ad recommendations faces latency bottlenecks and scaling ceilings when hybrid configurations process both long user histories and sparse feature interactions online. Meta addresses these trade-offs with a multi-stage architecture that decouples heavy, asynchronous offline user modeling from lightweight, real-time online ranking. Deep upstream transformer models process sequences of thousands of actions to generate cached user embeddings, which downstream online models combine with candidate ad features using dense tokenization and target-aware multi-head attention. This framework produces predictable, LLM-style log-linear scaling curves between compute and normalized entropy without disproportionate increases in online serving costs. Implemented as part of Meta's Generative Ads Recommendation Model, the design achieved a cumulative 6% lift in Instagram conversions and a 3% lift in Facebook conversions.


### [How Salesforce Eliminated Single-Region Risk and Reduced Downtime Blast Radius at 4B Metrics/Min](https://yomu.fyi/post/how-salesforce-eliminated-single-region-risk-and-reduced-downtime-blas.md)
- Company: [Salesforce](https://yomu.fyi/company/salesforce.md)
- Author: Scott Nyberg
- Published: Aug 5, 2026

Salesforce redesigned its internal observability platform, Argus, to eliminate single-region failure risks while ingesting approximately 4 billion metrics per minute. Operating out of a single AWS region previously created a global blast radius and generated high cross-region data transfer costs. To mitigate these risks without the prohibitive expense of full multi-region replication, the engineering team implemented a geo-local architecture that processes and stores telemetry closer to its origin across production geographies. A newly introduced federation query layer relies on Elasticsearch mappings to route queries selectively to relevant regional clusters, avoiding broad fan-out. The platform also adopted metadata caching for wildcard queries and implemented HTTP 206 partial-response handling to maintain visibility when individual regions experience downtime.


### [Removing the Security Barrier to Agentforce Adoption](https://yomu.fyi/post/removing-the-security-barrier-to-agentforce-adoption.md)
- Company: [Salesforce](https://yomu.fyi/company/salesforce.md)
- Author: Scott Nyberg
- Published: Aug 3, 2026

Enterprise AI adoption across regulated industries stalls when internal governance mandates private, dedicated network paths for sensitive data. To eliminate the need for manual VPN tunnels and complex proxy layers, Salesforce re-engineered Private Connect into a multi-cloud networking platform connecting customer environments to Hyperforce services. The Private Connect v2.0 architecture utilizes private links, transit gateways, and direct endpoint routing alongside an automated control plane for rapid Hyperforce regional rollouts. In addition to supporting cross-substrate interconnects for Azure and decoupling endpoint management, the platform incorporates a generic connector framework supporting tools like Snowflake, Databricks, and Kafka. Today, the system provisions secure connectivity in under 30 minutes while processing roughly 120 TB of data and 683 million requests monthly across 15 AWS regions.


### [GEM Training: How Meta Doubled the Efficiency of Its LLM-Scale Ads Foundation Model](https://yomu.fyi/post/gem-training-how-meta-doubled-the-efficiency-of-its-llm-scale-ads-foun.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Published: Aug 3, 2026

Meta's Generative Ads Recommendation Model (GEM) trains across thousands of GPUs using a hybrid architecture featuring trillions of sparse embedding parameters and billions of dense parameters. Standard training infrastructure built for large language models struggles with recommendation-specific workloads characterized by jagged user activity histories, asymmetric attention patterns, and numerical sensitivity. To overcome these bottlenecks, engineering teams introduced custom kernels like Jagged Flash Attention and Generalized Dot-Product Attention alongside MXFP8 mixed-precision execution. Pairing these kernel optimizations with topology-aware 5D parallelism doubled end-to-end training efficiency to 20–25% Model FLOPs Utilization while scaling total training FLOPs fourfold over twelve months.


### [How AI is transforming analytics at Grab](https://yomu.fyi/post/how-ai-is-transforming-analytics-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Maanas Prabhakar
- Published: Aug 1, 2026

Grab is restructuring its analytics operations using a five-level AI autonomy ladder, transitioning analysts from manual artifact creation to problem framing and decision governance. The architecture leverages domain-specific systems like Spartan to process natural language queries through certified metric indexes and Scarlet to triage and repair failing data pipelines. To prevent agent hallucinations, ContextIQ manages context lifecycles by automatically updating metric definitions, SQL references, and golden-dataset test cases when instrumentation changes or failures occur. Furthermore, data teams use an internal portal called BriX to configure custom analytics surfaces and automated root-cause analysis commentaries using reusable Model Context Protocol connections. Autonomy scales mechanical query and validation tasks while keeping human oversight focused on canonical metric definitions and strategic sign-offs.


### [Modeling Device Capabilities for Analytics](https://yomu.fyi/post/modeling-device-capabilities-for-analytics.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jul 31, 2026

Netflix supports diverse features, such as 4K streaming, immersive audio, and cloud gaming, across varied hardware platforms. Because hardware constraints like RAM, CPU cores, and display capabilities limit feature support on certain models, understanding device capabilities is crucial for managing user experiences. To address this challenge, Netflix built a comprehensive device capability data model integrated with internal feature flags. The architecture employs a cumulative table to capture the latest state of device capabilities alongside a histogram table tracking 28-day active device counts grouped by model and software version. These datasets power analytical products that evaluate feature reach and inform rollout decisions for capabilities like Netflix Spatial Audio and 4K Ultra HD.


### [GenRec: Towards LLM-Native Recommendation at Netflix](https://yomu.fyi/post/genrec-towards-llm-native-recommendation-at-netflix.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jul 30, 2026

Netflix developed GenRec, an LLM-backed recommendation ranker built by adapting an internal foundation model for large-scale personalization. Traditional recommendation stacks rely on thousands of hand-crafted features and specialized architectures that are costly to maintain across diverse content types. To replace manual feature pipelines, GenRec verbalizes user histories, metadata, and contexts into natural-language prompts and trains with multi-objective losses, including catalog-aware ranking and reward-weighted alignment. At inference time, the model executes in prefill-only mode on vLLM without decoding text. In large-scale online A/B testing against a mature production ranker, GenRec achieved statistically significant improvements in short-term and long-term metrics while using fewer labeled examples.


### [Crowdsourced taxonomy verification: A feedback-driven framework for refining knowledge graph relationships via online search interactions](https://yomu.fyi/post/crowdsourced-taxonomy-verification-a-feedback-driven-framework-for-ref.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Junpeng Niu
- Published: Jul 30, 2026

Maintaining accurate Knowledge Graphs in dynamic domains like e-commerce and food delivery is challenging because automated language models frequently hallucinate relationships while manual curation cannot scale. To validate structural taxonomy links continuously, a closed-loop verification framework operationalizes search interfaces by injecting unverified candidate edges as hypotheses into live user traffic. The system uses an exploration-exploitation strategy to place candidate relationships in lower-risk interface slots, tracking contextually anchored micro-interactions such as clicks, dwell times, and purchases. An offline verification engine aggregates these weighted interactions into normalized confidence scores, automatically promoting verified links to permanent graph edges and pruning refuted relationships.


### [Building Reliable Production AI with Durable Workflows](https://yomu.fyi/post/building-reliable-production-ai-with-durable-workflows.md)
- Company: [Salesforce](https://yomu.fyi/company/salesforce.md)
- Author: Scott Nyberg
- Published: Jul 27, 2026

AI prototypes rely on simple prompt-response interactions, but production AI systems handling thousands of records must manage distributed execution issues like worker crashes, rate limits, and transient failures. Rather than treating long-running operations as single large jobs or rebuilding state from logs, systems require durable workflows that preserve execution history across failures. In Agentforce Grid, execution is organized into a hierarchy where column runs divide into recoverable batches and rows composed of discrete activities. Each batch checkpoints after every successful row, retrying up to ten times with exponential backoff and resuming strictly at incomplete work to prevent duplicated model calls. Exposing progress across these hierarchical boundaries ensures that both users and engineers can distinguish completed, failing, and retryable units.


### [Agent platform (Part 1): How we help Grab build and run AI agents at scale](https://yomu.fyi/post/agent-platform-part-1-how-we-help-grab-build-and-run-ai-agents-at-scal.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Kendrick Tan
- Published: Jul 24, 2026

Grab scaled its internal AI tooling by transforming the infrastructure behind a single technical support bot into a standardized agent framework named LLM-Kit. The original Go-based bot revealed critical bottlenecks, including the lack of automated evaluation, painful provider migrations, fragmented observability, and excessive production scaffolding overhead. To address these frictions, engineers developed a FastAPI template pre-wired with LangGraph ReAct agents, remote Model Context Protocol servers, and built-in ROUGE, BLEU, and LLM-as-judge evaluation suites. The framework centralizes model access through an OpenAI-compatible GrabGPT Gateway and unifies observability across services, outbound calls, and agent steps using OpenTelemetry. This extracted scaffolding reduced day-one production setup time from two weeks to roughly an hour while supporting hundreds of production agents.


### [How AI Rebuilt Salesforce’s Decades-Old Localization Pipeline](https://yomu.fyi/post/how-ai-rebuilt-salesforce-s-decades-old-localization-pipeline.md)
- Company: [Salesforce](https://yomu.fyi/company/salesforce.md)
- Author: Scott Nyberg
- Published: Jul 23, 2026

Salesforce needed to deliver major releases across 34 languages as product localization volume surged by more than 35% under fixed budgets and unchanged delivery windows. To address these scaling constraints, the engineering team replaced their legacy translation workflows with an LLM-powered orchestration pipeline integrating prompt engineering, context engineering, and multi-stage validation. The architecture transforms existing style guides, glossaries, and terminology databases into structured context, routing interface strings through specialized prompt stages—up to 85 stages for some languages—to enforce product context, grammar, and branding rules. Combined with an automated validation loop spanning AI editing, AI validation, and human review, this system reduced localization costs by 50 to 90% while accelerating turnaround times across enterprise releases.


### [Personalizing Airbnb search by learning from the guest journey](https://yomu.fyi/post/personalizing-airbnb-search-by-learning-from-the-guest-journey.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Daochen Zha
- Published: Jul 21, 2026

Airbnb replaced hundreds of hand-crafted ranking features with a Transformer-based sequence model that captures both long-term booking history and short-term browsing behavior. To keep latency low and throughput high, sequence representations are generated via daily batch jobs and combined with real-time queries using a co-trained setwise ranker.


### [How our universal content processing platform Riviera evolved for AI and beyond](https://yomu.fyi/post/how-our-universal-content-processing-platform-riviera-evolved-for-ai-a.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash
- Published: Jul 20, 2026

Dropbox developed Riviera, a centralized content processing platform that evolved from an internal file preview generator into a shared infrastructure supporting over 300 file formats. By decomposing complex conversion tasks into modular, reusable transformations, the system now powers search indexing, media transcoding, and AI context extraction across Dropbox products.


### [In-House LLM Serving at Netflix](https://yomu.fyi/post/in-house-llm-serving-at-netflix.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jul 17, 2026

Netflix established an in-house serving platform to run large language model inference directly inside existing production environments alongside traditional machine learning models. Built upon NVIDIA Triton Inference Server and vLLM, the unified architecture handles member-scale routing, candidate generation, feature fetching, and model execution over gRPC and OpenAI-compatible HTTP endpoints. The team shifted their primary engine from TensorRT-LLM to vLLM to support custom architectures, simpler debugging, and non-trivial constraint logic. Addressing production obstacles required patching Triton's frontend for guided decoding, pinning dependent library versions, rewriting logits processors in C++ for vLLM V1, and handling state machine resets during engine preemptions. The resulting platform unifies deployment flows while preserving operational stability across evolving model schemas.


[Newer posts](https://yomu.fyi/topic/architecture.md) · [Older posts](https://yomu.fyi/topic/architecture/page/3.md)