API Versioning as a Translation Layer
API versioning as an architecture: requests enter through a version header, pass through a chain of reversible version changes, and reach controllers that only ever see the latest model.
Loading…
Topic
Architecture
830 posts about Architecture. Every summary links to the original.
- 830 posts
- 50 companies
- 31 explainers
See the idea
Architecture explainers
BIAN as a Contract Layer: Modernizing Core Banking One Domain at a Time
BIAN gives banks a shared vocabulary and service-domain contracts that sit between business concepts and the systems implementing them. See how domains, contracts, and governed access stack into one architecture.
Argo CD: The Reconciliation Loop Between Git and Your Cluster
Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Model Routing Is Simple. Until It Isn’t.
Model routing in agentic systems is presented as a systems optimization problem rather than a simple classification task. On 417 AppWorld Test Challenge tasks with the same CodeAct agent, Claude Sonnet 4.6 cost $79 total ($0.19 per task), versus $155 ($0.37 per task) for GPT-4.1, despite GPT-4.1 having lower listed token prices; cache-read pricing and repeated context changed the effective result. The discussion also shows why task difficulty and model speed are incomplete signals: hidden execution work, specialization, governance constraints, routing overhead, cache state, hardware, and endpoint load all affect decisions. The described router optimizes cost, quality, and latency together, producing configurable cost-accuracy operating points; one latency-oriented configuration reached 84% accuracy for $93 and 83 seconds, while adding about 6 ms and 2 kB per task.
Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization
Hierarchical Interest Representation is an upstream representation layer for Meta Ads that addresses sparse deep-funnel feedback by connecting users’ inferred interests with advertisers, products, services, and other ads entities. It models a typed, heterogeneous engagement graph with a transformer architecture, sparse attention for long-range relationships, bias-aware attention, self-supervised cross-view distillation, and multimodal content features processed through vision or language models. The system projects the raw graph into learned interest primitives and cascading super-graphs, producing universal embeddings and Bag-of-Meaning tokens at multiple granularities. Temporal cutoffs and chronological splits prevent future-information leakage, while a pipelined online graph engine reportedly achieved a 30x wall-clock speedup over a synchronous baseline during testing. Trained end-to-end on billions of interactions, the representations are designed for personalization, retrieval, supervision, and ranking integrations including GEM, Andromeda, and the Adaptive Ranking Model.
Welcome Inkling by Thinking Machines
Thinking Machines Lab has released Inkling and the smaller Inkling-Small family as open multimodal models that accept image, text, and audio inputs. Inkling is described as a roughly 1T-parameter decoder-only Mixture-of-Experts model, with 975B total and 41B active parameters, a 1M-token context window, and training on 45 trillion multimodal tokens. Its architecture combines relative attention, alternating global and sliding-window attention, short convolution, shared experts, hierarchical image patchification, and discretized mel-spectrogram audio processing. The release includes BF16 and NVFP4 checkpoints, speculative MTP layers, day-zero support in Transformers, SGLang, vLLM, and llama.cpp, plus deployment configurations and interactive voice and image demos. Reported benchmarks show Inkling and Inkling-Small achieving strong but varied results across reasoning, coding, vision, audio, factuality, chat, and safety evaluations.
@clickhouse/rowbinary: when your library is also a parser compiler
ClickHouse's @clickhouse/rowbinary is a Node.js reader and writer for the RowBinary, RowBinaryWithNames, and RowBinaryWithNamesAndTypes formats, offering both a generic parser and an accompanying Agent Skill. RowBinary avoids per-row overhead but is difficult to consume in JavaScript because nested types, LEB128 varints, precision-sensitive DateTime64 values, and recursive self-describing types require extensive runtime dispatch; JSON alternatives can also round large UInt64 values. The package provides small, composable, monomorphizable reading primitives, while SKILL.md instructs a coding agent to assemble query-specific parsers with inlined operations and selected policies such as BigInt handling and array materialization. Generated parsers run 1.5–3.4x faster than composing library functions in loops, with a fixed-width benchmark parser reaching 3.41x faster, and produce reviewable source that can be tested and benchmarked like ordinary application code.
Shipyard: How We Built Slack’s Next-Generation EC2 Platform
Slack developed Shipyard to modernize its EC2 infrastructure by replacing continuously updated, mutable instances with immutable deployable artifacts. The platform adopts a layered image architecture starting with a standardized base image called slack-zero, which is built using AWS Image Builder and consumed by downstream service-specific machine images. Rather than running periodic Chef jobs in the background, configuration management is restricted to image baking and initial instance provisioning phases. Fleet management is handled through the Peekaboo inventory system, metric-driven progressive rollouts via Gondola, and automated instance rotation managed by a lifecycle component known as the Reaper. Developers validate cookbook updates prior to merging using Ship Quick, which provisions real infrastructure across dedicated worker fleets to verify provisioning workflows.
Inside Shopify Hack Days: Building a prototype for music-playing pages
During Shopify Hack Days 39, a 13-person team addressed why artists selling music on Shopify product pages lacked a built-in way to hear digital albums before buying. The team pursued two paths in parallel: a first-class Audio media type across Core, Admin API, admin-web, and storefront, and a standalone Shop Sounds app for release management. The app orchestrated product and variant creation, audio and artwork metafields, storefront access, and Online Store publication, while a theme extension provided track-level purchasing and a shared audio instance that continued across navigation. It also included six GLSL fragment shaders driven by bass, mid-range, treble, and overall energy, plus synced lyrics and a geolocated tour dates block. After three days, the team had a working prototype, but Shop Sounds was not a live product; the project demonstrated that native audio support and Shopify metaobjects and metafields could support a music toolkit without custom infrastructure.
Training Orchestrator: Unifying Model Training at Yelp
Yelp developed Training Orchestrator to standardize machine learning pipeline execution and eliminate divergent training scripts across its applied engineering teams. Previously, monolithic Spark scripts coupled training logic directly to cluster runtimes, preventing local testing, slowing debug cycles, and causing inconsistent configuration drift across projects. The new framework introduces a declarative configuration layer backed by Pydantic models, enabling static type checking and step schema validation before runtime execution starts. Training workflows are defined as modular step hierarchies that the orchestration engine compiles into directed acyclic graphs for execution in topological order. By injecting Spark and MLflow contexts into steps, the architecture ensures reproducible tracking, automated Slack alerts, and parity across local development and production environments.
Building Service Topology at Scale: Architecture, Challenges, and Lessons Learned
Engineers at Netflix required a unified, real-time view of service dependencies to navigate distributed architecture and improve incident troubleshooting. Traditional batch systems introduced stale data, so the team created a streaming-first platform backed by reactive streams and backpressure handling to ingest flow records from multi-region Kafka streams and Server-Sent Events without data loss. The architecture partitions data into physically separate graph and columnar storage layers covering eBPF network flows, IPC metrics, and distributed traces. Network flow ingestion relies on a three-stage distributed aggregation pipeline using consistent hashing to resolve network intermediaries into logical application connections. The resulting production system serves time-travel and topology queries with sub-second latency while continuously updating dependency views.
Practical Guide To Python App Hosting
Python app hosting is presented as both an infrastructure choice and, for data-intensive or AI applications, a data-architecture choice. The guide defines a hosting stack comprising application code, runtime, dependency manager, application server and reverse proxy, and contrasts it with traditional web hosting. It compares shared hosting, VPSs and cloud VMs, PaaS, container platforms and serverless functions by control, operational effort, scalability and workload fit. It notes that long-running data and AI tasks such as model training, vector indexes, cached datasets, ETL jobs and lengthy inference pipelines can exceed practical serverless limits. The conclusion is that public APIs and lightweight web apps allow broad platform choice, while apps accessing governed data, model endpoints or AI agents should be evaluated by where data lives and how access is governed, with Databricks Apps presented as an option when data already resides in a lakehouse.
Introducing Feature Views
Feature Views are presented as a framework for creating managed feature pipelines in Databricks, addressing duplicated logic, training/serving skew, backfills, online-store plumbing, and governance for real-time ML. A Feature View defines the source, entity, time-series column, and computation once; Databricks uses it to generate point-in-time-accurate historical data, then materializes pipelines for batch or streaming production use. Registered in Unity Catalog, materialized features receive lineage and access controls, while MLflow records model dependencies and Model Serving retrieves required features without custom lookup code. For Kafka-sourced streaming features, Spark Realtime Mode, Lakebase, and Model Serving support rolling-window updates with reported end-to-end p99 latency of 200ms from event to online availability. The announcement says Feature Views are in Public Preview and can be developed through the Feature Engineering Client SDK and Genie Code, with streaming materialization requiring an Enterprise-tier workspace in a region supporting Lakebase.
AI Model Co-Design: Hardware-Friendly LLM Design
Efficient large language model serving requires co-designing architectures to balance datacenter token throughput and user interactivity within hardware constraints. Roofline analysis shows that linear layers become memory-bound when hidden or intermediate projection dimensions are small, leaving arithmetic hardware underutilized even at large token counts. For prefill workloads, Chunked Pipeline Parallelism reduces first-token latency on long contexts when models use regular, repeatable layer patterns that split into balanced stages. For latency-critical decoding, decoupling attention and feed-forward network parallelization enables independent optimization, applying expert parallelism for feed-forward networks and Helix parallelism to shard the key-value cache across sequences. Following architectural design rules—such as dimension alignment, width prioritization, and low-precision NVFP4 compatibility—allows models to achieve higher utilization on modern accelerator hardware.
A Practical Guide to GPU-Initiated Communication for Molecular Dynamics at Scale
Molecular dynamics packages like GROMACS face severe scaling bottlenecks when CPU-orchestrated MPI communication introduces blocking synchronizations during halo exchanges. To resolve this, researchers implemented GPU-initiated communication using NVIDIA NVSHMEM to eliminate CPU-GPU handoffs. The redesigned approach fuses packing, remote memory access, and completion waiting into device-side kernels using fine-grained signaling. Benchmarks on NVIDIA DGX H100 systems demonstrated up to 1.5x performance gains intra-node and up to 2x on GB200 NVL72 multi-node clusters, particularly on latency-bound systems. While device-initiated communication introduces a slight 1–3% overhead on very large compute-bound configurations at low node counts, it consistently improves strong scaling for production workloads.
GitFarm: Git® as a Service for Large-Scale Monorepos
At Uber’s monorepo scale, traditional Git workflows create bottlenecks through multi-gigabyte clones, local checkouts, repeated synchronization, and independent fetch or push operations across automation systems. The post describes GitFarm, a Git as a Service platform exposed through a high-performance gRPC API that executes standard Git commands in secure, ephemeral sandboxes backed by pre-warmed repository and container pools. Each backend maintains a bare clone, receives push-based updates and periodic git fetch operations every five minutes, while clients can explicitly fetch when they need current state. The design provides a full checkout in under 500 milliseconds and reduced client-side resource utilization by over 80%, while avoiding local clones and long initial clone delays.
amazon.science
Capturing token IDs during agentic interactions for better reinforcement learning
Reinforcement learning for agentic tasks requires updating model parameters against the precise token sequences generated during rollouts. Standard agent harnesses frequently modify conversation transcripts through message compaction and tool formatting, causing retokenization drift that degrades policy-gradient training. Turnstile addresses this issue as a Rust-based proxy situated between agent harnesses and inference backends such as SGLang. By intercepting OpenAI-compatible Chat Completions traffic, the proxy records exact token IDs, log probabilities, loss masks, weight versions, and mixture-of-experts routing traces at the moment of generation. Validation across text-based coding and multimodal computer-use agents demonstrated steady reinforcement learning improvements without requiring modifications to agent harnesses.
Modal is a computer
Modal is a platform that runs programs of arithmetic and logic operations on information, making it a computer. It operates on hardware from dozens of cloud providers, building a virtualization layer that aggregates, isolates, and multiplexes demand for increased resource efficiency. Programs run in containers that receive inputs from the container runtime, similar to how a typical computer’s ALU fetches inputs from memory. Information in Modal lives in a tiered storage hierarchy, from container memory to object storage, with caches like page cache and ephemeral disk. Input and output pass through the Modal Input/Output Plane for functions and sandboxes, or the Routing Plane for HTTP servers, with direct Tunnels for extreme performance needs.
Running Low-Latency Analytical Workloads with GPU-Accelerated Presto on NVIDIA GB200 NVL72
GPU-accelerated Presto integrates NVIDIA cuDF algorithms and NVLink to deliver low-latency analytical query execution on massive datasets. Benchmarks derived from TPC-H across scale factors 1K and 3K reveal that a single NVIDIA DGX B200 system with eight GPUs outperforms 8- to 10-node Intel Xeon clusters by up to 8.2x. Scaling out across multinode NVIDIA GB200 NVL72 systems paired with IBM Storage Scale achieved additional performance gains. Enabling NVIDIA GPUDirect Storage, tuning I/O parameters, adjusting thread counts, and applying query rebatching delivered a cumulative 64% reduction in query runtimes. Topology-aware GPUDirect Storage cold reads proved roughly twice as fast as standard POSIX reads by avoiding CPU memory staging and NUMA boundary crossing penalties.
How to Evaluate an Enterprise Analytics Platform
Enterprise analytics platform evaluations often overemphasize dashboard interfaces, although the larger decision concerns whether analytics, AI and agents share data, semantics and governance. The post distinguishes point solutions from a unified platform and proposes seven evaluation criteria: workload fit, architecture and openness, governance and compliance, performance and scalability, adoption and usability, AI and ML readiness, and total cost of ownership. It recommends mapping current and three-year workloads, testing production-scale data with realistic concurrency, measuring p95 latency, and examining governance, usability, contracts and operational complexity in a proof of concept. Lakehouse architecture, open formats such as Delta Lake and Apache Iceberg, and shared controls are presented as ways to reduce context gaps; Databricks is offered as a practical example using Unity Catalog, Genie and Agent Bricks. The conclusion favors a weighted, three-year assessment over a feature comparison.
NVIDIA Vera CPU Boosts AI Factory Throughput to Accelerate Agentic Workloads
Agentic AI workflows and reinforcement learning rely heavily on sequential CPU tasks, such as tool calls, sandboxed code execution, data processing, and environment rollouts between GPU steps. When CPU performance degrades under full socket load, training time inflates, serving latency grows, and GPU KV-caches face eviction. The NVIDIA Vera CPU addresses these bottlenecks by featuring 88 Olympus cores on a monolithic compute die, paired with power-efficient LPDDR5x memory and the Scalable Coherency Fabric. These cores incorporate a neural branch predictor, a 10-wide decode front end, deep out-of-order execution, and NVIDIA Spatial Multithreading. Consequently, Vera achieves 1.8x faster core execution, 40% lower peak loaded latency compared to x86 CPUs, and up to 1.2 TB/s total memory bandwidth, preventing costly KV-cache recomputations.
Maximize Spectral Efficiency with AI-Native RAN and NVIDIA AI Aerial
Massive MIMO deployments often fail to achieve theoretical spectral efficiency because CPU compute constraints force algorithmic compromises in user tracking, interference mitigation, and pairing. NVIDIA AI Aerial addresses this gap using a GPU-accelerated, AI-native radio access network architecture. This approach enables dense mathematical models across Layer 1 and Layer 2 workloads, including machine-learning beamforming and deep reinforcement learning link adaptation. In 64T64R simulations, AI beamforming achieved up to 1.62x higher throughput at 32 layers compared to regularized Zero Forcing baselines. The platform also unifies physical and MAC layer coordination while allowing dynamic allocation of spare GPU compute to edge AI inference.
Building an Analysis AI Agent for Industrial Alarm Management with NVIDIA Nemotron
Industrial machinery produces high volumes of alarms and sensor readings that exceed manual triage capacity. To address this bottleneck, an end-to-end GPU-accelerated AI agent is designed using NVIDIA NeMo libraries, Nemotron open models, and the NVIDIA OpenShell runtime behind a single HTTP endpoint. When triggered with an alarm payload and sensor frame, the agent gathers historical context using cuDF, cuVS, and NeMo Retriever, queries data warehouses via Text-2-SQL, and executes specialist signal checks with tools like cuFFT and cuML. Nemotron 3 Nano manages orchestration tasks while Nemotron 3 Super synthesizes the gathered evidence. The workflow outputs a structured package containing an observation, root-cause hypothesis, remedy, and recommended action within a seconds-long latency budget.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.