---
title: "Infrastructure"
description: "6 posts about Infrastructure, summarised, each linking to the original."
---

# Infrastructure
> 6 posts about Infrastructure, summarised, each linking to the original.

## Articles

### [How our AI Research team uses AI Runtime](https://yomu.fyi/post/how-our-ai-research-team-uses-ai-runtime.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Ethan Tang, Ivan Zhou, Irene Dea, Steven Chen, Bruce Fontaine, Ben Hansen
- Published: Oct 16, 2026
- Type: Problem & solution

Databricks’ AI Research team uses AI Runtime (AIR) to manage hundreds of parallel research workstreams for products like Genie, Smart Routing, and AI Functions, training models from a few billion to nearly a trillion parameters. AIR performs preflight checks on GPU nodes to ensure health and communication before training, and monitors node health during execution to detect issues like NCCL failures or thermal throttling. The AIR CLI and custom Docker containers enable fast experimentation by launching jobs in seconds without rebuilding environments, supporting both supervised fine-tuning and reinforcement learning. MLflow integration automatically tracks experiments, while Unity Catalog FUSE mounting gives governed access to research data across workspaces without changing training code. The team has scaled to agent-led training, where AI agents propose changes, launch trials, and evaluate results, increasing training job volume by 8x across AI Functions research workloads.


### [2026 World Cup: Breaking Records, Spiky Charts and Reliability at Scale](https://yomu.fyi/post/2026-world-cup-breaking-records-spiky-charts-and-reliability-at-scale.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: Syamla Bandla
- Published: Oct 15, 2026
- Type: Problem & solution

The 2026 FIFA World Cup produced the largest planned stress test of Meta’s infrastructure, with WhatsApp setting an all-time record of over 30 million messages per second during the final and delivering 8.4 billion messages in its peak five-minute window. Traffic analysis revealed that surges in sharing primarily occurred during breaks and final whistles, not during active gameplay, shaping capacity decisions. Instagram and Facebook saw live video watch time spike 90% and 242% respectively, while recommendation traffic rose 80% week-over-week. To absorb these spikes, Meta layered pre-provisioned capacity buffers, country-level graceful degradation, cascade prevention, and early warning signals across all serving tiers. Notably, the team did not shift load between regions or freeze deployments, relying instead on year-round reliability enhancements and phased runbook execution.


### [Modal crosses $1B in Annualized Run Rate and raises $750M Series D](https://yomu.fyi/post/modal-crosses-1b-in-annualized-run-rate-and-raises-750m-series-d.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Author: Erik Bernhardsson
- Published: Oct 14, 2026
- Type: Announcement

Modal crossed $1B in annualized run rate and raised $750M in Series D funding at a $15.75B post-money valuation, led by Accel. Usage has tripled since the Series C five months ago, with year-over-year growth exceeding 15x, and the company now powers applications from companies including DoorDash, Cognition, and Runway. To avoid the friction of the old stack, Modal built its own file system, container runtime, scheduler, and networking from scratch on a capacity pool spanning over 20 clouds. Sandboxes, launched in 2023 for running LLM-generated code, now represent a $300M run rate product with nearly 3 billion sandboxes spun up and scaling to a million concurrent. New products include multi-node training with a single decorator giving a fully RDMA-connected cluster, Modal servers for HTTP serving, Endpoint Candidates for LLM serving, and generally available virtual machines.


### [Rewinding the exponential: How we keep hardware usage down](https://yomu.fyi/post/rewinding-the-exponential-how-we-keep-hardware-usage-down.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Christian Endres
- Published: Oct 13, 2026
- Type: Problem & solution

Cloudflare’s Data team manages a pipeline handling over two billion events per second from tens of thousands of servers, powering Logs, SQL API, and observability. To counter exponential traffic growth against linear hardware expansion, they implemented several optimizations over five years, deferring two and a half years of hardware expansion. In 2022, they replaced a monolithic schema with product-specific schemas, reducing backbone bandwidth by 35-40%. In 2024, they switched compression from LZ4 to zstd, achieving a 40% reduction in compressed traffic with negligible CPU impact. In 2026, they decoupled pipeline transit from database redundancy via a shared ingest pool and late replication, applied sampling downstream, and introduced aggregations, eliminating about 70 GB/s of compressed traffic and halving projected 2031 data volume.


### [Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations](https://yomu.fyi/post/turning-opportunistic-ai-inference-capacity-into-better-ads-recommenda.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: Jinfu Leng
- Published: Oct 12, 2026
- Type: Problem & solution

Meta built a system that aggregates fragmented idle inference capacity from a globally constrained fleet into a shared opportunistic pool to enhance ads recommendations. Idle capacity arises from four mismatches: peak provisioning with demand variation, model development timing with capacity delivery, regional resource ratios, and accelerator types. Two elasticity levers exploit this preemptible, volatile capacity: Rank More Ads increases candidates evaluated by ranking stages, using both CPU and accelerator surplus, and Serve Better Models routes more requests to a more powerful model, using accelerator-only surplus. The system protects baseline serving through isolated routing, minute-scale feedback loops that adjust candidate volume and model coverage, and millisecond-scale value-aware load shedding that removes lower-value work during sudden preemption. This allows optional work to expand and contract with opportunistic capacity, driving better ad relevance and advertiser outcomes without requiring additional capacity orders.


### [How we scaled engineering enablement and infrastructure at Intercom - The Intercom Blog](https://yomu.fyi/post/how-we-scaled-engineering-enablement-and-infrastructure-at-intercom-th.md)
- Company: [Intercom](https://yomu.fyi/company/intercom.md)
- Published: Oct 5, 2026
- Type: Problem & solution

Intercom’s engineering enablement team transformed the company from a patchwork of legacy systems and recurring incidents into a stable infrastructure that empowers fast-moving product teams. The company learned that selecting core technologies is not enough without a clearly defined strategy for understanding, guiding their use, and investing in the teams behind them, as lacking this leads to persistent friction. They defined a dedicated core technology team, learned and standardized their tooling, and implemented approaches to hire and grow engineers who own the infrastructure. These efforts reduced operational friction and incident frequency, allowing the business to move more confidently. As a result, product teams run faster and the infrastructure is now a reliable foundation for growth.