Loading…
Latest reads
The engineering internet, summarised so you can actually read it.
Airbnb ·
Scaling beyond one: How Airbnb evolved its data architecture for a multi-product world
To support its expansion from Homes into Experiences and Services, Airbnb evolved its offline data warehouse architecture to balance domain-specific needs with organizational consistency. Rather than enforcing a single global pattern, data teams adopted a framework of foundational principles that paired separate data models for unique product features with monolithic models for cross-cutting domains like payments and messaging.
Patrick LamAmazon ·
Real-world grounding in agentic AI
Foundation models deployed as physical AI agents face critical risks when natural language hallucinations translate into physical law violations and operational hazards. To ensure physical consistency and reliability, researchers from the University of California, San Diego and Amazon Fulfillment Technology introduced four grounding pillars. Physics-guided deep learning embeds symmetries and differential equations into pretraining to obey conservation laws while reducing dataset requirements. The UQ4CT framework applies mixture-of-experts architectures to produce calibrated uncertainty estimates, cutting expected calibration error by over 25% and halting operations when safety thresholds are breached. Furthermore, the Adapting-While-Learning framework and formal verifiers like Hilbert integrate numerical simulators and interactive proof systems to enforce mathematical and scientific correctness.
Rose YuAmazon ·
Bridging intent and execution in agentic systems
AI agent performance increasingly depends on the harness mediating interactions between language models and tools rather than raw model reasoning. To address mismatches between model intent and harness execution, the authors developed Simple Strands Agent, an open-source single-agent harness. The architecture implements robust tool-interface practices, such as condensing middle log outputs from bash commands, enforcing line-aware string replacements with ambiguity checks, and returning diff files after edits for verification. Evaluated across benchmarks like SWE-Bench-Verified, SWE-Bench-Pro, and Terminal-Bench 2, these model-harness alignments yielded consistent performance gains across frontier models without task-specific tuning. The findings show that model families require tailored interface codesign alongside stable evaluation infrastructure to achieve optimal benchmark reliability.
Gaurav Gupta, Vatshank ChaturvediSentry ·
Works on my machine: how we use AI to reproduce reported bugs
Sentry maintains 159 open-source SDK packages across multiple languages, leading to high triage volume and significant boilerplate setup overhead when reproducing reported bugs. To streamline this workflow, the team developed a Claude skill that parses GitHub issue URLs to extract language, framework, and SDK versions. The skill generates a reproduction branch, sets up minimal boilerplate using ecosystem package managers like uv or npm, attempts execution, and opens a pull request with explicit run instructions. By allowing the model to exit early when encountering overly complex environments, the tool reliably prepares reproducible playgrounds without stumbling on edge cases. This workflow saved several hours in testing scenarios like Django HTTPS proxy bugs while avoiding excessive bot noise through scoped human interaction.
Neel ShahSentry ·
Errors, traces, logs, metrics: when to reach for what
Modern application telemetry relies on four overlapping signals: errors, traces, logs, and metrics. Each telemetry type addresses distinct engineering questions and powers specific debugging workflows. Errors track grouped, actionable application crashes, whereas traces map timing and execution paths across dependency waterfalls. Metrics aggregate numerical measurements across historical deployments, and logs preserve complete state and decision-making context at specific moments in execution. Emitting telemetry directly into the format needed for a given workflow proves superior to relying on a single wide event, as tools like AI coding agents require structured spans to optimize execution paths while human debuggers rely on unsampled logs and metrics to pinpoint silent failures.
Sergiy DybskiyAirbnb ·
Sitar-agent: Building a reliable dynamic configuration sidecar at scale
Airbnb modernized sitar-agent, a Kubernetes sidecar that delivers dynamic configuration updates to thousands of polyglot service instances without requiring redeployments. The architecture uses periodic AWS S3 snapshots to enable fast, decoupled pod startups and maintains local file-based storage for the main container to read configs with in-memory caching. Key design decisions include retaining an isolated sidecar over an in-process library and optimizing pull-based polling with server-side caching.
Bo TengMongoDB ·
Agentic Supplier Management with MongoDB Atlas, Voyage AI, and Multi-Modal Search
Retailers face major operational bottlenecks when managing supply chain disruptions through legacy enterprise resource planning systems and siloed documents. Traditional tabular structures struggle to index multi-modal data like images and PDF contracts, delaying critical supplier discovery during regional crises or tariff changes. To overcome these constraints, organizations decouple supplier management into a dedicated operational data layer using MongoDB Atlas and Voyage AI. This architecture stores polymorphic supplier profiles and high-dimensional vector embeddings within a unified collection, enabling natural language semantic search for alternate partners. Real-time updates propagate across systems using MongoDB Change Streams, allowing businesses to rapidly assess supply chain impacts and identify alternative vendors in minutes.
Ronan ConlonAmazon ·
Ground truth is a process, not a dataset
Evaluating search-augmented AI research reports requires verifying complex, multi-source claims that depend heavily on surrounding context. Unassisted PhD-level domain specialists tasked with establishing static ground-truth labels achieved only 60.8% accuracy on a hidden test set during controlled testing. To resolve benchmark errors on cognitively demanding tasks, Amazon AGI introduced the audit-then-score evaluation protocol alongside DeepFact-Bench and the DeepFact-Eval verification system. Under this protocol, model disagreements serve as challenges requiring concrete counterevidence and written rationales, enabling human auditors to evaluate competing claims and update benchmark ground truth prior to scoring. Operating under this iterative workflow, expert test accuracy rose to 90.9% across four rounds, and DeepFact-Eval reached 83.4% accuracy with GPT-4.1.
Venkatesh SaligramaAirbnb ·
When history fails you, borrow from geography
When historical travel data became uninformative during the asynchronous post-COVID recovery, Airbnb redesigned its demand forecasting to borrow signals across geographies rather than waiting for local data. Using a hierarchical Bayesian framework, the team propagated updated posterior estimates from early-recovering corridors as informative priors for structurally similar, later-recovering corridors. This allowed accurate corridor-level demand forecasting in near real time during periods of high disruption and uneven market reopening.
Harrison KatzGrab ·
From decentralized Docs-as-Code to a centralized repository: Evolving Grab's documentation strategy
Grab originally adopted a decentralized Docs-as-Code workflow to manage Markdown documentation alongside service code, but the model strained as engineering scaled. Dispersed repositories led to fragmented terminology, uncoordinated repository moves, uneven linting standards, and noisy search results. To resolve these issues, the organization transitioned to a centralized documentation repository with standardized templates, automated CI quality gates, and a unified search overlay combining fuzzy title search with Glean. Consolidating Markdown into a single repository also established a clean, structured knowledge base tailored for internal AI agents. The centralized setup fostered continuous maintenance, averaging 27 daily commits and an update merged roughly every 50 minutes.
Karen KueDropbox ·
Beyond code generation: rethinking engineering productivity in the age of AI agents
Dropbox shares how widespread AI code generation shifts software development bottlenecks downstream into code review, CI infrastructure, and validation pipelines. To adapt, they built Nova, an internal coding agent platform that safely automates scoped tasks such as migrations and flaky test remediation. They also evolved their developer productivity framework to measure end-to-end customer impact and code quality rather than simple pull request throughput.
Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki OkumuraSlack ·
Slack AI: The Path to Multi-Cloud
Slack evolved its Slack AI serving infrastructure across multiple phases to handle enterprise LLM workloads reliably and securely. The initial deployment on AWS SageMaker provided zero-knowledge escrow VPC isolation and FedRAMP compliance, but engineers faced scaling latency, GPU scarcity, and significant operational overhead. Slack then migrated live traffic to Amazon Bedrock to leverage managed Model Units and eliminate model release lag without customer-facing incidents. However, fixed Provisioned Throughput commitments and regional peak traffic variations created persistent underutilization challenges. Consequently, Slack expanded into a multi-cloud orchestration architecture that normalizes disparate provider APIs, integrates unified cross-cloud telemetry, and routes traffic dynamically around latency spikes and outages.
Shaurya KethireddyAmazon ·
How flat is replacing fat in AWS data center networks
Traditional data center networks rely on hierarchical fat-tree topologies, which introduce overhead, upper-layer congestion, and vulnerability to router failures. Although flat random networks theoretically maximize route diversity and resilience, routing computation and cabling complexity historically made them impractical for physical data centers. AWS addressed these constraints by introducing Resilient Network Graphs, a quasi-random architecture powered by a passive optical device called ShuffleBox and a routing algorithm named Spraypoint. Spraypoint sprays packets across random neighbors toward designated waypoints, unlocking twice as many independent paths while operating within commodity router memory limits. Adopted as the default architecture across new AWS facilities, this design reduces router counts by 69%, lowers projected network power consumption by 40%, and increases throughput by up to 33%.
Giacomo Bernardi, Ratul Mahajan, Seshadhri ComandurSentry ·
How we cut build times by two-thirds by deleting our CMS
Sentry rebuilt its marketing site and blog of roughly 2,500 pages, replacing a legacy Gatsby setup and traditional headless CMS with Astro, Markdown, Frontmatter, and AI-driven automation. The previous architecture suffered from 14-minute build times, restrictive CMS schema limits, and frequent build failures caused by external API dependencies. A team of 2.5 developers executed the migration in two months using Claude Code agents for data extraction, template translation, and testing. To manage content without a CMS dashboard, Sentry built Claude Skills to guide updates, run local previews, and draft GitHub pull requests. The migration reduced average build times to under 4 minutes, cut broken staging builds by 95 percent, and improved Web Vitals scores from 89 to 97.
Eli LennoxAmazon ·
Amazon Research Awards recipients announced
Amazon Research Awards has announced 70 award recipients across 49 universities in 11 countries for its fall 2025 funding cycle. The funded research spans six distinct tracks: AI for Information Security, Agentic AI, Automated Reasoning, AWS Cryptography, Cybersecurity and Anti-Abuse Technologies, and Sustainability. Each selected academic team receives unrestricted financial grants, AWS Promotional Credits to access specialized cloud AI and machine learning services, and access to more than 700 Amazon public datasets. Awardees are paired with an Amazon research contact who provides advisory support, consultation, and access to company training sessions. Furthermore, the program explicitly encourages recipients to publish their discoveries, deliver presentations at Amazon offices worldwide, and release associated software implementations under open-source licenses.
Amazon Research Awards teamDocker ·
Mitigating CVE-2026-31431 (“Copy Fail”) in Docker Engine
CVE-2026-31431, dubbed Copy Fail, is a Linux kernel vulnerability in the AF_ALG crypto subsystem that allows unprivileged users to perform controlled writes to the page cache. Because default Docker Engine profiles permitted AF_ALG sockets, containers on unpatched host kernels could corrupt shared host page caches. Docker's initial mitigation in v29.4.2 blocked AF_ALG sockets and the socketcall syscall via seccomp, but entirely blocking socketcall broke 32-bit binaries and Go runtimes. To resolve compatibility issues, Docker Engine v29.4.3 reverted the seccomp socketcall block and adopted Linux Security Modules to enforce socket restrictions. The updated release configures AppArmor and SELinux policies to block AF_ALG creation across both direct and multiplexed syscall paths while retaining seccomp filtering for defense-in-depth.
Paweł GronowskiSentry ·
You don’t need to pick one: how Sentry and OpenTelemetry work together
Integrating Sentry into systems already instrumented with OpenTelemetry does not require replacing existing backend SDKs or rewriting service instrumentation. A hybrid architecture pairs the Sentry SDK on the frontend for browser tracing, Session Replay, and logs with existing OpenTelemetry configurations across backend services. Frontend requests propagate W3C traceparent headers to backend endpoints, which then export traces and logs over OTLP directly to Sentry endpoints or through an intermediate OpenTelemetry Collector. Ingested telemetry attaches backend spans and standard Python logs to the initial frontend user actions, establishing a unified distributed trace across the entire request path. Although Sentry's OTLP ingest currently supports logs and traces rather than metrics, dedicated backend Sentry SDKs remain an optional addition later for backend exception tracking and profiling.
Lazar NikolovYelp ·
Beyond the Menu Tree: How Yelp Built a Smarter Customer Success Chatbot with AI
Yelp transitioned from a legacy customer support chatbot that relied on rigid phrase matching and static menus to a retrieval-augmented generation system. The updated architecture classifies incoming queries using a large language model to route users into one of five specialized workflows, including QA, Billing, Refund, Cancel, and Review. For general QA, the system uses FAISS to search an in-memory vectorstore built exclusively from article metadata such as titles, summaries, and headers rather than raw article text. Retrieved articles are supplied to the language model as prompt context, followed by automated validations for safety, character limits, and hallucinated hyperlinks. In production A/B testing, this metadata-driven design doubled the chatbot resolution rate while achieving approximately 94% recall@5.
Lina Lee, Machine Learning Engineer; Nelson Lee, Engineering ManagerSentry ·
Your agent can't fix what it can't see
Coding agents frequently fail to resolve production bugs when restricted to source code alone, as they lack critical runtime context such as stack traces, breadcrumbs, and environment data. To bridge this gap, Sentry introduces Sentry MCP and the Sentry CLI to supply agents and developers with structured production telemetry. The Model Context Protocol integration supports clients like Claude Code, Cursor, and VS Code, while the CLI enables scripted workflows. When an alert triggers, an agent uses trace and commit metadata to diagnose root causes across service boundaries, generate regression tests, and open draft pull requests. Over six months, the Autofix pull request merge rate increased from 41% to 46%.
Sergiy DybskiyAmazon ·
Diverse reasoning traces teach LLMs to make better decisions
Large language models typically rely on supervised fine-tuning with a single human-verified reasoning trace per example, which limits reasoning breadth and causes mode collapse when attempting parallel reasoning. Researchers presented set-supervised fine tuning (SSFT) and global forking policy optimization (GFPO) to teach models diverse problem-solving strategies. SSFT represents reasoning as a set of complete solution paths paired with global forking tokens, utilizing bipartite matching to achieve order-invariant training and specialized tokens. GFPO applies lightweight reinforcement learning to select the optimal forking token upfront, detaching gradients from generated rollouts to ensure stable optimization. On standard reasoning and coding benchmarks, the combined framework increased single-shot accuracy by 5% to 7% while improving pass@k diversity without degrading pass@1 accuracy.
Sheng Jia, Xiao Wang, Shiva Kasiviswanathan