API Versioning as a Translation Layer
API versioning as an architecture: requests enter through a version header, pass through a chain of reversible version changes, and reach controllers that only ever see the latest model.
Loading…
Topic
Architecture
830 posts about Architecture. Every summary links to the original.
- 830 posts
- 50 companies
- 31 explainers
See the idea
Architecture explainers
BIAN as a Contract Layer: Modernizing Core Banking One Domain at a Time
BIAN gives banks a shared vocabulary and service-domain contracts that sit between business concepts and the systems implementing them. See how domains, contracts, and governed access stack into one architecture.
Argo CD: The Reconciliation Loop Between Git and Your Cluster
Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
The foundations of Canva’s continuous data platform with Snowpipe Streaming
As Canva expanded to over 200 million monthly active users, ingestion throughput reached 25 billion records daily, causing AWS Data Firehose costs to consume nearly half of the product analytics platform budget. To reduce these expenses and eliminate intermediate file staging, the engineering team integrated Snowflake's Snowpipe Streaming directly with their Java-based Kinesis Data Streams pipeline. The architecture streams records directly into Snowflake tables using logical channels with offset checkpoints while configuring client buffering up to a five-minute maximum lag. In handling Kinesis Client Library edge cases, record processors coordinate with the LeaseCoordinator to drop leases when channels become unhealthy. Operating in production for over six months, the system ingested more than 20.35 petabytes of data, decreased query latency to roughly ten minutes, and reduced overall cloud spend by 45 percent.
Developing a modern architecture for energy utilities with embedded finance
Energy utilities can use embedded finance to create marketplace-like experiences for tariffs, solar installations, smart-home devices, storage, and related services while capturing more energy-related revenue. The proposed foundation combines Stripe Online Payment Flows, Billing, Payments, Connect, and Data Pipeline with AWS serverless services, including Lambda, EventBridge, DynamoDB, Redshift, S3, and QuickSight. A tariff enrollment or payment can trigger webhooks and real-time event processing, while transaction and customer data support dashboards, usage insights, personalized recommendations, subscriptions, and broader partner offerings. The architecture also describes PCI Level 1 security, encryption, Radar, Strong Customer Authentication, AWS IAM, KMS, Shield, WAF, Config, Security Hub, and CloudWatch for compliance and monitoring. It concludes that this integrated payment, analytics, and security foundation can help utilities improve customer engagement, support sustainable programs, and develop new revenue streams.
How do I store inventory data in my Stripe application
The DevRel Swag Store uses Stripe payments and AWS services to keep product inventory accurate and visible in near real time. Stripe products retain core payment information, while Amazon DynamoDB stores inventory and other attributes, keyed by Stripe product ID and store ID; EventBridge routes payment events to Lambda, which atomically decrements stock and publishes updates through AWS IoT Core to the frontend. DynamoDB condition expressions prevent decrements when stock is insufficient, while post-payment validation and refunds address the delay before a Stripe Payment Link is disabled, although Stripe fees may be non-refundable. A custom Payment Intent flow can check stock before payment, authorize funds with manual capture for up to seven days, update inventory after payment success, and then capture the charge.
The science of routing print orders
Canva's global print network requires selecting optimal suppliers to balance delivery times, packaging counts, and environmental emissions. To resolve these challenges before user checkout, the engineering team designed a modular routing architecture that decouples graph construction, decision logic, and path traversal. During graph traversal, the system generates action objects capturing forward paths and decision query results, which are compiled into timestamped routing logs in blob storage for asynchronous auditing. Utilizing preprocessed graph queries alongside ElastiCache, Redis, and database read replicas, the infrastructure sustains high-throughput evaluation without coupling cost logic to traversal code. As a result, print routing completes within an average of 50 milliseconds at the 99th percentile during peak usage while maintaining 99.999% data availability.
Growing your Stripe integration With Event Destinations
Stripe Event Destinations, now in general availability and rolling out to all accounts, lets companies extend payment workflows without changing existing integrations. The post explains how Stripe events can be sent either to an AWS account through Amazon EventBridge or to a webhook endpoint, with EventBridge routing selected events to services such as AWS Lambda, Step Functions, and SQS. Event filtering, logging, transformation, and built-in security reduce the need to operate webhook servers and verify signatures in application code. Serverless Lambda functions can add custom logic and connect to DynamoDB, S3, or SNS, supporting examples including date-limited memberships, partial student payments, and real-time tipping broadcasts through AWS IoT Core and WebSockets. The resulting architecture is presented as flexible and scalable for variable or growing event volumes while preserving the underlying Stripe integration.
Managing SaaS access control with Stripe’s Entitlements API
Managing access rights across multiple SaaS pricing plans can require maintaining feature lists, subscription and pricing relationships, and related database resources as plans or features change. The post presents Stripe’s Entitlements API as a way to register features, associate them with Stripe Products, and retrieve active entitlements for a customer’s subscriptions. Its example defines Free, Personal, and Business monthly products priced at $0, $98, and $980, then maps article-viewing and business-report features to those products using Stripe CLI commands in Workbench. Customer entitlement checks can support application authorization, while entitlements.active_entitlement_summary.updated webhooks expose before-and-after permission data for workflows triggered by plan changes or cancellations.
How we reduced peak memory and CPU usage of the product configuration management SDK
GrabX’s product configuration SDK used substantial memory and CPU because every client downloaded, decoded, and loaded a single JSON file containing configurations for all services, even when it needed only a small subset. The design also regenerated and fetched complete files every minute, while decoding a file larger than 100 MB could starve request handling and increase P99 latency. The proposed redesign partitions data by service, stores each configuration in its own JSON file and S3 prefix, and adds service-level changelogs so SDKs subscribe only to required services and apply targeted updates. Benchmarking across varied configuration sizes found potential reductions of up to 70% in memory utilisation and more than 50% in maximum CPU utilisation. Further work is planned to reduce SDK initialisation time.
How We Built the BFCM 2023 Globe
The 2023 BFCM Globe is a real-time visualization of purchases made through Shopify-powered merchants worldwide, with animated arcs, city dots, fireworks, camera movement, atmospheric noise, stars, and a custom material. Performance was a central concern because the earlier approach used a separate mesh and draw call for every arc, causing more than 1,000 simultaneous arcs to slow some mobile devices. The revised arc system uses instancing, interleaved per-arc attributes, and vertex-shader Bézier deformation, allowing tens of thousands of arcs in one draw call and an initial benchmark of up to one million arcs on an M1 laptop. Other effects were similarly consolidated, including billboarded point particles for cities and single-draw-call fireworks, while 2D psrdnoise and a stripped-down shader reduced rendering cost. The result combines richer visuals with substantially greater rendering capacity.
Improving Compute Sustainability: A Case Study
Two Sigma's large computing footprint drives significant energy consumption and carbon emissions, particularly across live trading applications that require continuous real-time market data caching. To address this overhead without sacrificing performance, engineering teams used a routine hardware refresh to transition from legacy single-process machines to denser multi-core server configurations. By replacing roughly 60 legacy hosts with 28-core processor hardware, the team distributed baseline power draw over more cores and eliminated underutilized compute capacity. This architectural consolidation reduced absolute power consumption across production hosts by 66%, dropping electricity usage from 27 MWh in January 2023 to 10 MWh in January 2024. The initiative subsequently established an annual sustainability rationalization practice for hardware budgeting across the organization.
Evolution of Catwalk: Model serving platform at Grab
Grab developed and scaled Catwalk, an internal machine learning model serving platform, to address operational bottlenecks, low resource utilization, and deployment friction between data scientists and backend engineers. The platform transitioned from an admin-managed TensorFlow Serving setup into a low-code self-service system supporting PyTorch and ONNX, before replacing complex Helm charts with Kubernetes Custom Resource Definitions for declarative, blue-green deployment orchestration. To support complex business workflows and multi-model applications, Grab subsequently introduced Catwalk Orchestrator with bundled deployments that allow individual services to scale independently. Across two years, the orchestrator architecture expanded to 200 deployed applications serving approximately 1,400 production machine learning models.
Choosing the right sandbox strategy for your organization
Stripe sandboxes provide isolated environments for testing Stripe features without affecting a live integration, with access controls for the people who use them. Each live account can have up to five sandboxes, which admins can create, name, configure by copying selected live-account settings, and delete when no longer needed. The guide describes a staged setup for startups and small teams, using separate environments for development, integration testing, user acceptance testing, and spare capacity, while larger organizations may assign sandboxes to departments and rotate them between projects. It also presents an iterative model in which teams prototype, collect stakeholder feedback, retest changes, and document results. The recommended strategy depends on team size, organizational structure, workflow, and development methodology.
Bringing Grab’s Live Activity to Android: Enhancing user experience through custom notifications
Grab designed an equivalent to iOS Live Activities for Android to provide real-time order tracking outside the app. Because Android lacks Apple's native ActivityKit push token system, the team substituted push tokens with placeholder values to maintain technical consistency across platforms and preserve backend push targeting via their Hedwig service. For the user interface, engineering selected custom notifications over floating views because custom notifications avoid intrusive screen usage and do not require the 'Draw over other apps' permission. The client implementation separates responsibilities across dedicated classes: LiveActivityIntegrationManager handles token registration across business verticals, LiveActivityAttributes encapsulates UI configuration, and LiveActivityManager maps payloads to Android NotificationManager instances. The solution launched for Food, Mart, Express, and Transport verticals.
Unveiling the process: The creation of our powerful campaign builder
Grab details the event processing architecture behind Trident, its internal marketing campaign platform that evaluates If This, Then That (IFTTT) logic over Kafka streams. The core processing unit is a treatment consisting of an event, optional conditions, and actions. Complex campaign capabilities—such as counters, limits, and delays across multiple hours via recursive SQS message scheduling—are assembled from multiple coordinated treatments. To simplify campaign creation, Grab introduced a flowchart-like visual builder represented as a JSON node tree that compiles recursively into treatments while persisting node-to-treatment mappings to reconcile edits over time.
Tinder API Style Guide — Part 1
Tinder’s rapid growth and expansion into domain-centric teams produced inconsistent API practices across services, including mixed error conventions, fragmented schemas, uneven versioning, unclear URIs, and weak API contracts. Part 1 presents standards for the RESTful APIs used across the platform, beginning with the URI pattern https://service-mesh-host-name/[version]/[service-name]/[resource]/[sub-resource] and guidance on concise, plural, kebab-cased paths. It defines path-prefix versioning, retaining the previous version until clients migrate and creating new versions only for breaking changes such as removing fields, changing types, or renaming fields. The guide also covers HTTP methods, standard and custom request headers, the X-Tinder- naming convention, and request tracing through MDC-populated identifiers and centralized logging. Request and response bodies, status codes, and enforcement tools are reserved for Part 2.
Iterating with shadow workspaces
Cursor’s shadow workspace addresses the problem of letting AI agents iterate on code without disrupting a developer’s active workspace. The design prioritizes LSP usability—especially lints, definition navigation, and language-server interaction—while also requiring independence, privacy, concurrency, universality, maintainability, and speed. A copied in-memory TextModel was rejected because sharing the user’s language server could introduce copied files into references, duplicate declarations, or miss errors. The implemented approach uses a reusable hidden Electron window for the workspace, routing AI edits through Cursor’s processes and returning lints independently of the visible editor. The post treats runnability as a harder future extension, with folder-proxy designs, platform limitations, and network-level independence still open questions.
Peeking under the hood of Stripe Invoicing
Stripe Invoicing’s no-code workflow hides the API calls and invoice state transitions that developers may need to inspect when debugging payment failures. Using Stripe Workbench’s Inspector, Logs, and Events tabs, the walkthrough maps object hierarchies, requests, event payloads, and previous_attributes across an invoice lifecycle. Creating an invoice in the Dashboard produces a draft Invoice, a separate Invoice Item, an automatically generated Invoice Line Item, and related creation or update events. Sending it creates a Payment Intent and finalizes the invoice from draft to open; charging captures the Payment Intent, creates a Charge, and changes the invoice to paid. Draft invoices can be deleted, while open invoices can be voided, preserving the record and canceling the Payment Intent, and the same Workbench method can be applied to other Stripe API areas.
Building an Event-Driven Architecture with Managed Data Services
Event-driven architecture (EDA) addresses the need to process and respond to continuous streams of real-time activity from applications, e-commerce systems, wearables, and other IoT sensors. Using Apache Kafka, independent producers publish events to topics while consumers subscribe and trigger downstream processes, keeping services decoupled and independently scalable. Kafka also distributes large data streams in real time and replicates data across synchronized broker nodes, so a node failure does not lose data or interrupt operation. Apache Kafka on Heroku is presented as a fully managed service, with multi-tenant basic and single-tenant private plans, automatic broker scaling, and Standard-tier-and-above configurations starting with three brokers and extending to eight. Recommended integration practices include mapping producer-consumer flows, using transactions, topic and schema management, and delivery guarantees, plus monitoring performance and logging Kafka operations.
LLM-powered data classification for data entities at scale
Grab needed to classify sensitive data at the table and column level across petabytes of database tables and streaming schemas. Manual schema-tiering campaigns had resulted in half of all schemas receiving overly strict Tier 1 access controls, while an initial automated service using regex patterns and third-party machine learning produced high false-positive rates and lacked customizability. To address this, the Caspian data engineering and governance teams enhanced their internal orchestration service, Gemini, by integrating GPT-3.5 via Azure OpenAI. Gemini aggregates classification requests into mini-batches, handles API rate limits, and uses prompt engineering—including few-shot examples, curated tag libraries, and explicit JSON DTO schemas—to reliably tag columns for data owner verification.
Office Hours with Engineering Managing Director Mae Santos
Mae Santos, Head of Critical Applications and Data Reliability Engineering at Two Sigma, leads a global team across New York, Houston, London, and Tokyo to support critical applications and data pipelines. While software teams frequently prioritize functional features over operational stability, designing systems with reliability in mind from the beginning allows for progressive enhancements without complete rewrites. System architecture should be guided by concrete goals across four key dimensions: availability, observability, scalability, and supportability. In addition, mapping out interconnections among systems, infrastructure, data, and human processes helps teams manage dependencies effectively. Applying automation across support and delivery workflows further reinforces reliability by minimizing human error and accelerating execution.
No version left behind: Our epic journey of GitLab upgrades
Grab's self-hosted GitLab instance fell approximately 14 months behind official releases, serving thousands of engineers and monorepos reaching up to 39TB in total footprint. To address security updates and system stability, the team established a structured upgrade routine using Terraform, Packer, and Ansible across GitLab's 5,000-user reference architecture. They implemented staged deployments, upgraded stateful Gitaly nodes via in-place rotation, and resolved primary node clustering bottlenecks under Praefect. By utilizing official GitLab Upgrade Paths and parallelizing component deployments, the team caught up on 24 months of releases in 11 months and reduced release lag from 396 days down to 35 days.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.