API Versioning as a Translation Layer
API versioning as an architecture: requests enter through a version header, pass through a chain of reversible version changes, and reach controllers that only ever see the latest model.
Loading…
Topic
Architecture
830 posts about Architecture. Every summary links to the original.
- 830 posts
- 50 companies
- 31 explainers
See the idea
Architecture explainers
BIAN as a Contract Layer: Modernizing Core Banking One Domain at a Time
BIAN gives banks a shared vocabulary and service-domain contracts that sit between business concepts and the systems implementing them. See how domains, contracts, and governed access stack into one architecture.
Argo CD: The Reconciliation Loop Between Git and Your Cluster
Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
How Indra unified EV charging data on Databricks
Indra Renewable Technologies consolidated a fragmented EV-charging data estate spanning Cosmos DB, Synapse, Azure Data Lake Storage, Azure Functions, Power BI, and other services onto Databricks. The migration began with a parallel Synapse prototype and expanded to Delta tables, a consolidated fleet pipeline, and greater use of Databricks for operational workloads. A medallion architecture stores raw fleet data in bronze, transforms it with PySpark in silver, and publishes gold tables for reporting and client delivery; one pipeline now serves three clients and improved performance by 60% to 70%. Unity Catalog, serverless SQL warehouses, AI/BI dashboards, and Genie Agents support governed self-service analytics, while reported results include 80% to 90% business cost savings, 90% storage savings, and query latency falling from 24.4 seconds to 3.5 seconds.
How AI-Powered Attacks Led Salesforce to Reinvent Hyperscale DDoS Defense
Salesforce built DREAM (DDoS Response and Mitigation) to defend its distributed cloud against hyperscale, machine-speed DDoS attacks, with a stated goal of responding in seconds while protecting more than 4.5 million domains and 3 million customer orgs at 99.99% availability. The platform targets rapidly evolving application-layer attacks in a shared, multi-tenant environment by combining AI-driven inference with distributed orchestration for telemetry, policy deployment, notifications, retries, and fault-tolerant execution across regions. The team selected Temporal rather than rebuilding distributed-systems primitives, and used AI coding assistants to migrate about 100,000 lines at roughly ten times prior velocity; more than 90% of the rewrite was AI-assisted, with human reviews and test coverage above 95%. Production lessons moved bulk data outside orchestration and bounded workflow history, while DREAM reduced time to first mitigation fivefold.
MAPS: Netflix’s Multimodal Asset Personalization at Scale
Netflix treats artwork and video previews as personalized assets, but ID-based models struggle with newly launched titles and assets that have little interaction history. The systems encode artwork with 768-dimensional CLIP image embeddings and combine those vectors with learned asset IDs, allowing visual preferences to transfer across titles and enabling one model to serve five canvases. Reward-based weighting balances pooled canvas data by the long-term value of interactions, while inverse propensity scoring on randomized exploration traffic supports offline evaluation. For video previews, MediaFM combines visual, audio, and text signals and ranked ahead of SeqCLIP in a linear probe, offline IPS evaluation, and online A/B testing; the Netflix Embedding Store supplies consistent embeddings to production models.
Self-service backup for VMs and containers on OpenShift: No cluster-admin required
In multi-tenant Red Hat OpenShift clusters, Velero Backup and Restore resources live in the OADP installation namespace, leaving non-admin application teams dependent on platform staff. OADP self-service addresses this with namespace-scoped NonAdminBackup and NonAdminRestore resources that proxy Velero resources, while a controller validates requests against DataProtectionApplication policies and synchronizes phase, progress, and queue status back to the tenant namespace. Administrators can enforce retention, snapshot and file-system backup settings, resource inclusion or exclusion, and approval for tenant-provided backup storage; cluster-scoped resources such as SCCs, ClusterRoles, CRDs, and KubeVirt cluster preferences are automatically excluded. OADP 1.6 supports the same workflow for containers, stateful applications, and OpenShift Virtualization VMs, but self-service has no native schedules, cross-namespace restores, or short-lived credentials for tenant-provided storage locations.
AI didn't write our SDK. It changed how we built it.
Stripe's extensibility SDK was built with AI involved across prototyping, architecture, implementation, and refinement, but the main change was to the development loop rather than code-generation speed. For a custom-object design, one engineer produced a working prototype, demo, and documentation site in roughly two days instead of an estimated two to three weeks, allowing the team to evaluate boundaries and reprioritize the feature for the next major launch. The team then used reviewed, detailed specifications and persistent domain guidance to delegate bounded work, including about twenty ESLint rules covering authoring constraints such as sandbox boundaries, imports, and type safety. AI also helped keep scripts, workflows, SDK generation, runtime enforcement, tooling, documentation, examples, and tests aligned, while human judgment remained responsible for abstractions and trade-offs. The post concludes that AI's leverage came from making more artifacts concrete earlier, enabling earlier feedback and a tighter deliverable.
So, is ClickHouse winning the observability wars?
The article examines whether ClickHouse is “winning” observability by separating the storage-and-query question from the broader work of collection, schema design, correlation, visualization, alerting, and investigation. It argues that ClickHouse fits telemetry because its columnar layout stores fields separately, enabling compression and allowing analytical queries to read only selected columns across millions or billions of events; ordering keys, sparse primary indexes, and data-skipping indexes further reduce work. The discussion covers both rich, wide events and conventional logs, traces, and metrics, and presents ClickStack as a combination of ClickHouse, an OpenTelemetry Collector, and a UI that preserves familiar workflows. The conclusion is qualified: ClickHouse has become the de facto storage and query engine for many teams and a strong default for large telemetry workloads, but it has not won observability as a whole, while full-text indexes, PromQL support, and broader product layers remain important.
Building for the AI Era: Lakebase, Streaming, and Lakehouse Innovations at VLDB 2026
Databricks previews its VLDB 2026 keynote, four accepted papers, a demo, and a sponsor talk covering database architecture, streaming, and Lakehouse optimization. Reynold Xin will introduce Lakebase, which separates serverless PostgreSQL compute from storage, and LTAP, a transactional-analytical model intended for AI-agent workloads; Lakebase stores data and write-ahead logs in cloud object storage and supports sub-second cold starts and copy-on-write branching. The papers describe a decade of Spark Structured Streaming evolution, including microbatch pipelining that improved throughput by up to 3x, stateful APIs, and fine-grained access control, alongside AutoLiquid’s CLUSTER BY AUTO and Ultron’s history-based query optimization. AutoLiquid outperformed customer-selected clustering keys on over 95% of evaluated workloads, while Ultron improved median join latency by 25% in production workloads. The program also includes an Enzyme demonstration of incremental materialized-view maintenance and a sponsor presentation on LakehouseRT and the Reyden engine.
What QSR reports miss about the decisions matter the most
Traditional QSR reporting can show that a limited-time offer (LTO) missed systemwide expectations, but not whether the cause was weak demand, lower franchise participation, ingredient shortages, inconsistent execution, slower service, or menu cannibalization. The post presents Lovelytics’ QSR Executive Performance Control Tower, an AI blueprint/solution built on Databricks that connects corporate performance, supply chain and operations, customer and digital, and financial signals. It is designed to distinguish demand from operational and participation issues, link changes to likely drivers and financial impact, and give corporate and franchise leaders a daily briefing on what changed, why it matters, and where to focus. The stated advantage is faster, more actionable decision-making based on a governed, reusable intelligence layer rather than disconnected reports.
Vertical Advantage: Transforming Industries with Lakebase and Agentic AI
The post presents partner-built, industry-specific solutions on Databricks Lakebase for financial services, manufacturing and energy, retail and consumer sectors, travel and hospitality, healthcare and life sciences, communications and media, and the public sector. Lakebase is described as a serverless Postgres transactional database on the Databricks Data and AI Platform, combining operational and analytical workloads under unified governance and providing operational serving, agent memory, audit trails, and transactional runtime capabilities. Examples include regulation-change monitoring, AI-native claims operations, KYC and personalization accelerators, near-real-time fraud detection, telecom customer-lifecycle decisioning, and public-sector emergency Situation Reports, using components such as Unity Catalog, Lakeflow, Genie, Mosaic AI agents, and Agent Bricks. The stated outcome is production-ready acceleration from insight to action, including faster compliance response, claims adjudication in minutes, low-latency decisions, and reduced manual effort, with partners positioned to help scope pilots.
Object Storage + WAL: Lakebase Postgres for the agentic era
The post presents Lakebase Postgres as a design for agent workloads that need isolated copies, restores, replicas, historical queries, and many short-lived databases without repeatedly moving large datasets. It contrasts conventional data-centric OLTP with a transaction-centric model in which PostgreSQL’s write-ahead log (WAL) is the source of truth, with LSNs making database history addressable. Lakebase splits compute, which runs standard Postgres and uses RAM and local NVMe caches, from storage, where safekeepers replicate WAL through quorum acknowledgements, a pageserver materializes pages, and object storage retains immutable history. Commits wait for replicated WAL acknowledgement, not page uploads, while reads request pages at an LSN. The design supports point-in-time queries, cheap branch or read-only compute, scale-to-zero suspension, and shared transactional and analytical access, while still relying on RAM and NVMe for performance.
llm-d flow control: Priority queuing for shared GPU inference
Shared GPU model pools allow platform teams to maximize utilization, but mixed workloads often contend for serving capacity with differing service requirements. Generally available in Red Hat AI Inference 3.5, llm-d flow control introduces explicit admission control prior to backend routing. An Endpoint Picker sits in the request path and evaluates active priority bands from highest to lowest integer values whenever pool saturation is detected. Saturation metrics combine per-replica queue depth and key-value cache pressure to determine when dispatch gates should engage. Once a priority band is admitted, tenant fairness policies distribute dispatch turns, and ordering rules select the next request to forward to a vLLM replica via Envoy.
A universal interface: How QuintoAndar made ClickHouse plug-and-play with managed Postgres
QuintoAndar’s customer data platform was split between a Postgres-based hot layer for low-latency lookups and a Databricks-based cold layer for analytical processing, creating duplicate code and expensive MERGE INTO operations. The team consolidated the pipeline by having its event gateway write directly to ClickHouse Cloud, which also serves API requests and analytical queries while remaining accessible to Trino, Spark, and Superset. Hightouch’s Customer Studio did not support ClickHouse, so QuintoAndar used ClickHouse Managed Postgres and the pg_clickhouse extension to expose ClickHouse tables as Postgres tables. Queries are pushed down to ClickHouse, while Hightouch’s planner and audit tables remain in NVMe-backed Postgres, allowing the existing activation workflow to continue with negligible round-trip impact for that use case.
NVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure
AI factories need custom XPUs to handle larger models and complex reasoning workloads, but scaling them requires HBM bandwidth, package and silicon area, power efficiency, supply-chain resilience, and rack-scale integration. NVIDIA NVLink Fusion addresses this by connecting custom XPUs and CPUs to NVIDIA’s AI infrastructure platform, using its scale-up and scale-out stack, ecosystem, and MGX rack-scale architecture. Its package-level NVHBM custom HBM base die, designed and validated with leading memory vendors, provides up to 30% more bandwidth and 15% lower HBM power usage than standard HBM4e, while enabling up to 25% more compute die area. The combined NVLink Fusion and NVHBM design is described as delivering a 30% overall end-to-end performance increase per XPU and simplifying integration into heterogeneous, rack-scale deployments.
How to Train a Cross-Embodiment Robot Navigation Policy with AI Agents
The tutorial presents an agent-driven workflow for adapting robot navigation across embodiments and scenes, addressing the repeated cost of collecting data, preparing simulation assets, training, diagnosing, and evaluating each robot-scene pair. Using Boston Dynamics Spot, it applies COMPASS to a built-in warehouse and a SAGE-10K scene, with NVIDIA Omniverse NuRec described as an option for captured environments. COMPASS reuses the pretrained NVIDIA X-Mobility navigation policy and trains a residual reinforcement-learning specialist that corrects base actions for a selected robot and environment; specialists can later be distilled into a shared policy. Repository skills guide dependency checks, asset preparation, smoke testing, training, checkpoint comparison, and packaging, while human gates control scene acceptance, smoke-test continuation, and checkpoint promotion. Runtime integration converts camera images, goals or routes, and speed from odometry into velocity commands, with cuVSLAM available separately when compatible odometry and transforms are unavailable.
Gallup scales real-time coaching for thousands with Amazon Bedrock
Gallup needed to centralize decades of proprietary workplace performance research to provide proactive, real-time coaching for organizational leaders directly within their daily workflows. To solve this challenge, Gallup developed Gallup AI inside its Gallup Access platform using Amazon Bedrock and Anthropic Claude models. The multi-tenant serverless architecture leverages Amazon Bedrock Knowledge Bases and Amazon Kendra for retrieval-augmented generation, while AWS Lambda with FastAPI powers real-time streaming responses. Amazon ElastiCache Serverless caches recent conversation history for sub-millisecond session retrieval, Amazon RDS for MySQL provides durable transactional storage, and Amazon Bedrock Guardrails enforce content safety. Since its June 2024 launch, the system has achieved sub-second time-to-first-byte streaming latency while prompt volume grew approximately sevenfold across thousands of organizations.
Closing the AI agent trust gap with graduated autonomy
Operating AI agents under binary full-access or read-only permissions creates an operational trust gap due to unpredictable model behavior across updates. The graduated autonomy architectural pattern closes this gap by dynamically granting or revoking tool permissions through sustained reliability scoring. A scoring engine computes a composite score across five weighted dimensions over a rolling 50-action window, mapping agents across four tiers from probation to full autonomy. In-process pre-execution filters check for prompt injection and sensitive targets, while Amazon Bedrock AgentCore Gateway and Cedar policies strictly enforce deny-by-default access at the infrastructure level. Post-execution state capture in Amazon DynamoDB supports reversibility, and AWS CodePipeline delivery gates block releases failing adversarial evaluation benchmarks.
Experiment with Qwen3.8-Flash-Next on NVIDIA GB300 NVL72 for Agentic Coding
Alibaba released the model weights for Qwen3.8-Flash-Next as a preview of the upcoming Qwen4 architecture, enabling developers to experiment with a multimodal mixture-of-experts model with a 125B-parameter main model, 51B N-gram embeddings, and 6B parameters activated per token. It supports a native 262,144-token context window, extensible to 1M tokens with YaRN. To reduce long-context attention compute and KV-cache growth, its hybrid design uses Gated DeltaNet in three of every four layers and Qwen Sparse Attention in the remaining layer, where micro-block importance guides retrieval. On NVIDIA GB300 NVL72, it delivers more than 16K tokens per second per GPU and more than 200 tokens per second per user; QSA achieved up to 7.6x prefill and 4.9x decoding speedups versus full attention. Fine-tuning is available through NVIDIA NeMo AutoModel with SFT or LoRA, and reinforcement learning through NVIDIA NeMo RL.
Scaling Conditional Learned Retrieval for Pinterest Home Feed
Pinterest home feed candidate generation relies on large-scale retrieval, but standard two-tower models produce single embeddings that struggle to capture diverse concurrent user intentions. To solve this, Pinterest expanded Conditional Learned Retrieval (CLR) to generate condition-aware embeddings conditioned on explicit interest, Pin, and Board contexts. The team evolved model representations by appending condition tokens to user sequences in transformers, integrating the PinFM foundation model, and introducing hierarchical semantic IDs to improve cold-start coverage. On the serving side, engineers consolidated duplicate feature requests and shifted inference to GPU instances using NVEmbed under a one-query-to-many-documents paradigm. These infrastructure and modeling changes unlocked new condition types, achieved seven-figure cost savings, and reduced p90 model retrieval latency from 80ms down to 12ms.
How Ubuntu Is Using Rust to Rebuild Core System Tools
Canonical is selectively adopting Rust to rewrite core system utilities across Ubuntu releases to improve memory safety, security, and long-term resilience. Starting in Ubuntu 26.04 LTS, uutils coreutils shipped as the default implementation targeting complete GNU compatibility, alongside sudo-rs, which introduced intentional behavior changes like asterisk display on password entry. Canonical is also funding ntpd-rs to consolidate NTP, NTS, and PTP into a single utility, while collaborating on UPKI to bring certificate revocation to system utilities. To manage Rust packaging at distribution scale, Canonical vendors dependencies per package rather than creating individual Debian packages for crates and embeds software bills of materials into binaries using cargo auditable. Future plans include exploring Rust implementations of compression libraries like bzip2-rs, zlib-rs, and zstd-rs.
Leveraging Databricks to Support FISC Security Guidelines
The FISC Security Guidelines define control objectives for Japanese financial institutions, while the post addresses the challenge of translating them into auditable controls on Databricks data and AI workloads. The Databricks Data + AI Platform, Unity Catalog, and optional Enhanced Security Monitoring and Compliance Security Profile features are presented for identity and network controls, encryption, fine-grained access, audit logs, lineage, and hardened runtimes. For AI workloads, Unity Catalog’s Model Registry links training data, code versions, and evaluation metrics to model versions, while a separate framework maps 97 AI-specific risks to 73 controls. The FISC Customer Capabilities Mapping Matrix connects guideline units to platform capabilities and customer responsibilities, including configuration, SIEM integration, backup, recovery, and evidence; the Security Analysis Tool is suggested for finding workspace gaps, and the material is identified as educational guidance requiring expert review.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.