API Versioning as a Translation Layer
API versioning as an architecture: requests enter through a version header, pass through a chain of reversible version changes, and reach controllers that only ever see the latest model.
Loading…
Topic
Architecture
830 posts about Architecture. Every summary links to the original.
- 830 posts
- 50 companies
- 31 explainers
See the idea
Architecture explainers
BIAN as a Contract Layer: Modernizing Core Banking One Domain at a Time
BIAN gives banks a shared vocabulary and service-domain contracts that sit between business concepts and the systems implementing them. See how domains, contracts, and governed access stack into one architecture.
Argo CD: The Reconciliation Loop Between Git and Your Cluster
Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Indexing the Data Lake for Online Point Queries
Online services and AI agents require low-latency point queries over massive datasets that are economically impractical to store entirely in key-value systems like Bigtable or DynamoDB. While cloud object storage offers rapid request latency, traditional distributed query engines like Trino and BigQuery introduce seconds of planning and scheduling overhead along with dependent file reads. Random Access Parquet addresses this limitation by using an external index mapping keys directly to exact Parquet files and row locations. Readers execute O(1) index lookups and issue parallel ranged reads to fetch only the required bytes without scanning footers or metadata. By running directly against existing data lake files, this architecture eliminates separate storage copies and allows shared analytical data to serve interactive online lookups.
Six Agent Harness Capabilities for Higher Model Performance
NVIDIA Labs released NVIDIA Labs Object-Oriented Agents (NOOA), an open-source research preview treating AI agents as single Python classes. The harness implements six model-facing capabilities: typed inputs and outputs, passing live objects by reference, code-based actions, programmable loops, explicit object state, and model-callable harness APIs. Instead of context compaction, passing live references keeps transcripts append-only, preserving prompt cache validity across long sessions. NOOA also includes a curated SQLite-backed long-term memory system forming a typed knowledge graph. Across evaluations, NOOA achieved 82.2 percent on SWE-bench Verified with GPT-5.5 using roughly 1.1M tokens per task, 86.8 percent on CyberGym L1 without network access, and 85.1 percent mean RHAE on ARC-AGI-3 using GPT-5.6-sol at under twenty dollars per game.
AI customer service: strategy, agents, and solutions guide
AI customer service combines natural language processing, machine learning, predictive analytics, generative AI, and automation to route requests, generate responses, and support resolution across channels. The guide distinguishes AI agents from scripted chatbots: agents reason over context, call external tools, and complete multi-step tasks such as checking shipping data and issuing a partial refund. It presents AI as a layer that absorbs routine volume while human agents retain judgment over complex or sensitive cases, citing potential gains including more than 70% query automation, up to 30% lower operating costs, and a 15% improvement in customer satisfaction. It recommends evaluating integration, decision transparency, production support, security, and compliance, then starting with one measurable, high-volume use case and expanding only after consistent resolution quality.
ModelExpress: Distributing Model Artifacts at the Speed of Light
Distributing large language model weights and JIT kernel caches across clusters imposes high latency on cold starts, autoscaling, and reinforcement learning loops. NVIDIA ModelExpress minimizes this delay by identifying existing compatible weight replicas across the cluster rather than treating every worker as an independent cold start. Initial replicas bootstrap by streaming directly from remote object storage into GPU memory or bypassing host staging using GPUDirect Storage. Subsequent replicas load pre-processed weights and kernel caches directly from serving peers over peer-to-peer RDMA via the NVIDIA Inference Xfer Library. In testing on DeepSeek-V4 Pro, this peer-to-peer artifact distribution transferred weights and caches in under ten seconds, reducing overall replica startup time from eight minutes to one minute and forty-four seconds.
How AI Rebuilt Salesforce’s Decades-Old Localization Pipeline
Salesforce needed to deliver major releases across 34 languages as product localization volume surged by more than 35% under fixed budgets and unchanged delivery windows. To address these scaling constraints, the engineering team replaced their legacy translation workflows with an LLM-powered orchestration pipeline integrating prompt engineering, context engineering, and multi-stage validation. The architecture transforms existing style guides, glossaries, and terminology databases into structured context, routing interface strings through specialized prompt stages—up to 85 stages for some languages—to enforce product context, grammar, and branding rules. Combined with an automated validation loop spanning AI editing, AI validation, and human review, this system reduced localization costs by 50 to 90% while accelerating turnaround times across enterprise releases.
Treating Pricing Changes Like Code Deploys
Thumbtack manages marketplace pricing by updating hundreds of thousands of bid settings stored in DynamoDB across service offerings. Previously executed through individual custom Go scripts, these pricing rollouts lacked automated runtime state tracking, dynamic database throughput pacing, and durable records for reliable rollbacks. To eliminate manual operational risk, engineering introduced a shared deployment-like engine that accepts declarative target states and executes pure Go transformation functions without direct I/O. The engine enforces idempotency, conducts dry runs with validation reports to S3, coordinates atomic DynamoDB updates with audit logging, and dynamically manages write capacity. This platform standardizes pricing updates into reviewable, resumable, and reversible operations that safely skip already written rows and check for production drift during rollbacks.
Building a serverless AI assistant at Pelago: concept to care in two weeks
Pelago engineered a serverless, event-driven AI assistant on AWS within two weeks to provide substance use recovery coaches with contextual response suggestions. Operating under strict Protected Health Information constraints, the system maintains clinical safety through human-in-the-loop review and keeps data confined to an Amazon VPC. Incoming messages flow via AWS AppSync and Amazon DynamoDB to an Amazon SNS topic, which fans out to independent AWS Lambda functions. The asynchronous Chat Assistant Lambda queries history, invokes Amazon Bedrock, and persists suggestions into Amazon RDS MySQL in under 10 seconds. When coaches access a conversation via Amazon API Gateway, pre-generated suggestions load in under 100 milliseconds.
Securing Infrastructure at Scale: Introducing Pinterest’s Resource Provisioner Pipeline (RPP)
Managing Infrastructure as Code across distributed repositories presents significant security and operational challenges. Pinterest developed the Resource Provisioner Pipeline (RPP), a proprietary Terraform execution engine, to securely deploy and manage AWS infrastructure across hundreds of workspaces. The pipeline utilizes centralized GitHub composite actions triggered by pull request events and enforces dual human approvals. To maintain least privilege, RPP uses GitHub OpenID Connect authentication to assume a central identity before validating repository paths and down-scoping to specific team execution roles. Automated checks include backend state validation, linting, plan generation, Semgrep static analysis, and optional LocalStack testing before deliberate comment-driven deployments execute.
Building multi-Region resiliency for AWS CloudFormation custom resource deployment
AWS CloudFormation custom resources extend template provisioning capabilities but lack built-in multi-Region coordination, risking duplicate executions and unhandled failures during regional outages. To solve this limitation, an active-active architecture leverages Amazon Simple Notification Service topics to fan out lifecycle events across primary and secondary infrastructure Regions. The primary Region processes events immediately, whereas the secondary Region processes incoming messages after a deliberate delay implemented via Amazon Simple Queue Service. Coordination relies on Amazon DynamoDB Global Tables with conditional writes to manage distributed locking and enforce idempotency. When failures occur in the primary Region, Amazon CloudWatch alarms signal Amazon Application Recovery Controller to automate failover without manual intervention.
Architecting offline-first generative AI applications for edge deployments using AWS services
Industrial edge environments often face significant unplanned downtime yet lack reliable cloud connectivity to run large-scale artificial intelligence models. To address this limitation, an offline-first reference architecture shifts generative inference to edge devices while leveraging cloud services for preparation and lifecycle management. The workflow uses Amazon Bedrock with Amazon Nova Pro to create labeled dataset pairs, Amazon SageMaker AI Pipelines to fine-tune a compact model, and AWS IoT Greengrass to orchestrate edge deployments. Local execution relies on Ollama, Strands Agents, and a CPU-based ChromaDB retrieval pipeline that consumes zero GPU VRAM. In evaluations across three foundation model judges, the fine-tuned gpt-oss-20b model combined with retrieval-augmented generation consistently outperformed the base model configuration across accuracy, completeness, and relevance metrics.
Automate custom PII detection at scale with Amazon Macie and Step Functions
Regulated organizations ingesting large data volumes must detect and classify standard and domain-specific personally identifiable information to satisfy compliance mandates like GDPR, HIPAA, CCPA, and PCI DSS. To eliminate manual inspection, an event-driven architecture orchestrates Amazon Macie scans for files landing in Amazon S3. Amazon EventBridge captures upload events and invokes AWS Step Functions, which manages Lambda functions to stage objects, trigger Macie classification jobs with custom regex identifiers, and poll for results. The pipeline separates data across a three-bucket pattern representing raw, staged, and scanned states to prevent mixing unprocessed data with validated assets. Finally, the workflow exports timestamped compliance reports in CSV and JSON formats while alerting security teams via Amazon SNS on high-severity findings.
Session revocations at scale
Canva gateways check in-memory session revocations on every request, but pulling over a million revocations from MySQL during deployments created severe database stampedes. To eliminate this bottleneck, the team transitioned to storing a twelve-hour sliding window of revocations in Amazon S3 partitioned into 30-minute chunks. Each revocation is packed into a 16-byte binary structure within a sorted flat array, allowing gateways to query records via direct binary search without object deserialization. Asynchronous workers batch updates into S3 using ZooKeeper leader election and conditional PUT requests for optimistic concurrency control. This architecture reduced the in-memory cache footprint by 87.5% and allowed the team to reduce their MySQL database to just two read replicas.
The last mile: why great first-party data still doesn't make great marketing
Modern data platforms and sophisticated marketing systems can still fail to produce timely customer experiences when no bridge connects first-party data to campaign activation. The post explains Scott Brinker’s “composable canvas,” a five-ring architecture centered on a unified data core, with semantic layer, CaaS, decisioning, and apps and agents operating on shared data without repeated movement. In contrast, batch files, dashboards, and disconnected teams can leave autonomous agents unable to trigger campaigns or propensity models unused. It recommends closing this last mile through activation-ready data architecture, self-service marketing analytics, and one narrowly scoped AI agent tied to a measurable campaign outcome. Examples in the post include a 4X model conversion rate, a four-week account scoring launch, and grocery offer flows that changed weekly preparation from hours to minutes.
Zone-Failure-Resilient OpenSearch® at Uber
Uber describes how it made OpenSearch deployments zone-failure resilient so querying and ingestion remain available after a complete zone loss. The design combines OpenSearch’s shard-allocation awareness with Odin-based isolation groups that distribute nodes predictably across failure domains and balance roles. Clusters maintain at least three shard copies across three isolation groups, while forced awareness keeps shards for a missing group unassigned instead of triggering heavy rebalancing; this trades immediate full replication for stability in surviving zones. For manager resiliency, five cluster manager nodes and cluster.auto_shrink_voting_configuration allow quorum after a zone failure plus one additional manager failure. Uber reports that isolation groups replaced uneven physical zone IDs, enabling complete shard assignment, preventing disk skew and hot nodes, and supporting green cluster health.
Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72
Frontier AI pre-training increasingly relies on mixture of experts architectures, which lower compute per token while introducing severe all-to-all communication bottlenecks. To address these scaling limits, the NVIDIA GB300 NVL72 rack-scale system co-designs Blackwell Ultra GPUs, fifth-generation NVLink scale-up fabrics, and 800 Gbps SuperNIC scale-out networking. Testing on DeepSeek-V3 671B across 256 GPUs achieved a pre-training benchmark of 1,648 TFLOPs per GPU using Megatron Core. In addition to hardware co-design, ongoing software optimizations across Megatron Core, TorchTitan, and JAX contributed major throughput gains over time. Scaling from 256 to 1,024 GPUs sustained over 97% per-GPU performance efficiency across all evaluated frameworks by overlapping scale-out gradient traffic behind compute windows.
Inside NVIDIA Rubin GPU Architecture: Powering the Era of Agentic AI
Agentic artificial intelligence workloads require sustained multistep inference, demanding low per-step latency, high decode throughput, and efficient long-context processing. NVIDIA designed the Rubin GPU to deliver up to ten times more agentic throughput per unit of energy than Blackwell. The hardware combines 336 billion transistors across two dies unified by NV-HBI, up to 288 GB of HBM4 memory providing 22 TB/s peak bandwidth, and the third-generation Transformer Engine reaching up to 50 petaflops of NVFP4 compute. Performance optimizations include inline descriptor updates for the Tensor Memory Accelerator, doubled K-dimension instruction throughput, 3-bit lookup-table weights, and structured 2:4 sparse attention compression. At the rack level, Vera Rubin NVL72 leverages Intelligent Power Smoothing and DSX MaxLPS to enable up to 40% more GPUs within a fixed power budget.
NVIDIA Vera CPU: Olympus Cores Built for Maximum Single-Thread Performance in Agentic AI
Agentic AI workloads shift critical execution paths onto CPUs through sandboxed environments that execute code, invoke tools, and manage runtime states. The NVIDIA Vera CPU addresses these compute requirements via Olympus cores built to maximize single-thread performance and instructions per cycle under full socket load. To handle irregular control flows and pointer-heavy structures, Olympus combines a ten-wide decode engine, neural branch prediction supporting two taken branches per cycle, dependency-breaking acceleration, and dedicated graph prefetchers. Across scale-up systems, dual-socket Vera configurations leverage second-generation NVLink-C2C interconnects to maintain a clean two-NUMA-node topology alongside PCIe 6.4, CXL 3.1, and Arm CCA confidential computing. These architectural choices provide predictable latency and memory bandwidth across highly concurrent reinforcement learning and agent loops.
Personalizing Airbnb search by learning from the guest journey
Airbnb replaced hundreds of hand-crafted ranking features with a Transformer-based sequence model that captures both long-term booking history and short-term browsing behavior. To keep latency low and throughput high, sequence representations are generated via daily batch jobs and combined with real-time queries using a co-trained setwise ranker.
Building a Transformer-Based Category Recommender at Thumbtack
Thumbtack developed a transformer-based category recommender to personalize email and push notification suggestions across roughly 500 home service categories. Because user activity is sparse while contextual signals like location and climate are rich, the architecture represents search history and context features as sequences of tokens evaluated against candidate category tokens using cross-attention. Initial training with naive cross-entropy suffered from severe popularity bias, skewing recommendations toward head categories and yielding a low tail hit rate. To resolve this issue, the team implemented adaptive mixed sampling, combining in-batch and random negative candidates and keeping only top-scoring hard negatives for cross-entropy loss computation. This method raised tail hit rate from 2.1% to 28.4%, improved overall HR@10 to 54.6%, and yielded a statistically significant 5% lift in requests per user in production A/B testing.
NVIDIA NVLink: The Scale-Up Network for AI Factories
Scale-up networking has emerged as a critical architectural component for AI factories supporting trillion-parameter models, mixture-of-experts architectures, and disaggregated serving. NVIDIA presents its sixth-generation NVLink scale-up fabric and NVLink 6 Switch to address latency-sensitive all-to-all GPU communication and offload collective operations. In simulation benchmarks on models like DeepSeek-R1, Qwen 235B, and a simulated 2T parameter model, NVLink achieved up to 2.3X higher decode throughput than off-the-shelf Ethernet in a 72-accelerator domain. The architecture integrates NVLink-C2C to supply 1.8 TB/s of coherent CPU-GPU bandwidth in the Vera Rubin NVL72 platform, as well as NVLink Fusion for integrating semi-custom silicon XPUs. Through co-designed hardware, SHARP in-network compute, and NCCL libraries, the platform optimizes token throughput, factory uptime, and deployment economics.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.