API Versioning as a Translation Layer
API versioning as an architecture: requests enter through a version header, pass through a chain of reversible version changes, and reach controllers that only ever see the latest model.
Loading…
Topic
Architecture
830 posts about Architecture. Every summary links to the original.
- 830 posts
- 50 companies
- 31 explainers
See the idea
Architecture explainers
BIAN as a Contract Layer: Modernizing Core Banking One Domain at a Time
BIAN gives banks a shared vocabulary and service-domain contracts that sit between business concepts and the systems implementing them. See how domains, contracts, and governed access stack into one architecture.
Argo CD: The Reconciliation Loop Between Git and Your Cluster
Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Unsupervised graph anomaly detection - Catching new fraudulent behaviours
Fraud detection systems relying on historical training data often struggle to detect new fraudulent modus operandi because emerging patterns lack labeled supervision. Grab developed GraphBEAN, an unsupervised autoencoder model designed to detect anomalous patterns across bipartite interaction graphs representing consumers and merchants. The architecture uses graph convolution layers to encode node and edge attributes, then reconstructs features and edge existence through separate feature and structure decoders. High reconstruction errors flag rare, anomalous behaviors at both the node and edge levels. An automated pipeline applies heuristic fraud tags to these scores and routes them to human analysts and automated mitigation systems.
What I learned taking Ramp Bill Pay from 0 to N
An engineer recounts lessons from taking Ramp’s Bill Pay from its first code in April 2021 to a publicly launched product that eventually became a product with multibillion annual total payment volume. For 0-to-1 work, the account emphasizes protecting core abstractions, including canonicalized payment statuses that reduce varied payment states to five, using mocked functions to coordinate parallel engineering, and reaching beta customers quickly. Customer feedback, session replay, and persistent iteration are presented as necessary because the first customer will usually expose substantial flaws. For 1-to-N growth, manual processes and duct tape must be prioritized for replacement according to user impact, error risk, and workload. Growing stakes also require testing, monitoring, contingency plans, gradual rollouts, QA, and documentation that helps engineers self-service onboarding.
Shopify’s platform is the Web platform
Shopify now recommends Remix for building Admin apps, which merchants use to configure apps, view data, and integrate services within Shopify Admin. Because these apps run in cross-origin iframes, browser restrictions can block cookies, storage, URL updates, and direct access to the Admin interface; App Bridge uses postMessage() to restore capabilities and support deeper integration. The new App Bridge client replaces the previous package, requires only a script tag, and makes standard web APIs such as history.pushState() work with Admin apps. The Remix template uses file-based routes, server-side loaders and actions, implicit API handling, and Shopify-specific integrations supplied by @shopify/shopify-app-remix, with Prisma provided for storage and sessions. Shopify integrated the template into Shopify CLI and describes it as the beginning of broader platform and design-system reworks.
Contributing support for a Wasm instruction to Winch
Adding a WebAssembly instruction to Winch, Wasmtime’s baseline non-optimizing compiler, requires coordinated changes across code generation, register management, ISA-specific emission, and tests. The contribution described adds visitor support, delegates through CodeGenContext and MacroAssembler, emits x86_64 instructions through Cranelift-backed assembler methods, and registers the instruction for differential fuzzing. CodeGenContext keeps the value stack, register allocator, and function frame consistent, while spilling can move values to linear-memory offsets; MacroAssembler handles ISA differences, register constraints, and CPU feature fallbacks such as lzcnt versus bsr-based clz. Filetests inspect generated machine code without executing it, while differential fuzzing compares Winch results with another WebAssembly engine across randomly generated modules and inputs. The workflow also includes running fuzzing, incorporating draft-PR feedback, rebasing, and submitting the public Wasmtime pull request.
Moving Fast by Moving Slow: How We Built Payments at Ramp
Ramp describes why it delayed the launch of BillPay by about two months to replace its existing, ad hoc bank-transfer infrastructure with a new payments service. The earlier system encoded product-specific concepts such as employee bank accounts and reimbursement transfers, forcing each new use case to add bespoke data models, jobs, and scheduling logic. This friction became especially costly as the team anticipated additional payment products and vendor payments scheduled for future dates. The redesign used generic primitives—ACHDetails and TransferCanonical—to separate business logic from money-movement logic; it also treats failed transfers and retries as concerns requiring an auditable identity, history, and consistent provider mapping. The post argues that stress-testing assumptions, modeling relevant degrees of freedom, limiting conditional logic, and building platforms after recurring patterns can make future products faster to launch.
Zero traffic cost for Kafka consumers
Grab's real-time data streaming platform team reduced infrastructure expenses by enabling Kafka consumers to fetch from the closest partition replica instead of partition leaders. The platform previously operated across three AWS Availability Zones with three-way replication, resulting in cross-AZ network fees that constituted half of the Kafka platform's total cost. To address this, the team performed a zero-downtime upgrade to Kafka 3.1, configured a rack-aware replica selector using AWS Availability Zone IDs, and updated consumer SDKs to populate the client rack parameter. The rollout produced a 25% drop in cross-AZ traffic within three months under steady data volumes. However, the change increased end-to-end latency by up to 500ms, created potential CPU load skews across zones, and removed graceful isolation during broker rotation maintenance.
Playing Traffic Cop with Fly-Replay
Fly.io’s fly-replay response header lets the Fly Proxy replay an HTTP request on another app, region, or VM within the same organization. The post explains how this can send writes near a leader database, preserve request locality after uploads, wake Machines stopped on exit, or route customer traffic across apps. It then builds a small Go reverse-proxy application that looks up the request hostname in SQLite and returns either app=our-default-app or the registered customer’s app in the header. Local request handling and lookup took about 4 ms, while the real-world proxy-and-replay path added about 100 ms; private customer apps remained reachable because Fly services can be defined without public IP addresses. The post also mentions LiteFS and internal DNS as possible discovery mechanisms.
Next steps for Postgres pluggable storage
PostgreSQL pluggable storage would let different tables in one database use storage engines optimized for transactional, analytical, or archival workloads, while allowing developers to build engines in an extensions-like model. PostgreSQL 12 added basic support through refactoring and expanding table access method APIs, initially to enable ZHeap, whose undo log showed promising results and addressed long-term table bloat, but work on ZHeap and TAM appears stalled. The current API has important gaps, including limited index flexibility, all-or-nothing index updates, and row-version identifiers tied to block and offset numbers; available options such as Citus Columnar Storage remain constrained by these limits. OrioleDB's upstreaming has reduced its fork's patchset from about 5,000 lines in PostgreSQL 14 to about 2,000 for PostgreSQL 16, raising the possibility that required changes could land in PostgreSQL 17.
Message Center - Redesigning the messaging experience on the Grab superapp
Grab redesigned its messaging infrastructure from GrabChat to Message Center to overcome two-party chat limitations and support complex superapp requirements like group conversations and varied user roles. The architecture separates core processing logic from message delivery by splitting the system into a backend processor and an independently scalable postman service. Communication relies on an in-house TCP gateway named Hermes that proxies client payloads via gRPC, alongside Apache Kafka streams and Amazon SQS delay queues. Custom client-server acknowledgements and a DynamoDB event store ensure reliable message delivery even across dropped mobile TCP connections and offline reconnects.
Creating a Flexible Order Routing System with Shopify Functions
Shopify’s smart order routing replaces a one-size-fits-all approach to choosing which inventory location fulfills an order with configurable routing rules. Rules can rank eligible locations by proximity to the destination, staying within the destination market, or merchant-defined location priorities, and merchants can arrange them in an order in the admin. Each rule is a Shopify Function written in Rust, receives data shaped through a GraphQL query, and returns location rankings; a reducer resolves ties by applying later rules to locations tied at rank 0. The rebuilt process uses the pipeline pattern to separate input preparation, rule execution, reduction, and final location selection, improving readability and debugging compared with the earlier tightly coupled implementation. This first stage dogfooded Shopify Functions on its own merchant stores, with a later stage planned to let developers add more complex rules.
Supabase Logs: open source logging server
Supabase is releasing Supabase Logs for self-hosted users and CLI development, built around Logflare, which processes more than 1 billion log events daily from Supabase services. Logflare is a multi-node, highly available Elixir cluster that ingests events through Vector or native integrations, streams them to BigQuery, and exposes SQL-backed HTTP Endpoints for Studio, Logs Explorer, and usage charts. Endpoints support parameterized queries, in-memory read-through caching, configurable cache warming, and select-only query sandboxing, allowing views over billions of daily events without querying many terabytes on every request. The release changes Logflare’s license to Apache 2.0, replaces closed-source SQL parsing with rust-based sqlparser-rs, and improves environment-based configuration for self-hosting; BigQuery is currently the supported backend, with more planned.
How Migrating from Vanilla Redux to Redux Toolkit Improved State Management in Shopify POS
Shopify’s React Native Point of Sale team migrated its production state management from Vanilla Redux to Redux Toolkit (RTK) after developers found boilerplate and debugging difficult. They rejected a full rewrite in favor of incremental reducer-by-reducer conversion, allowing Vanilla reducers and RTK slices to coexist and limiting rollback and destabilization risk. Most reducers adopted RTK’s mutable syntax and generated action creators, while asynchronous logic retained a hybrid thunk approach because RTK Query did not support GraphQL and wholesale use of pending and fulfilled actions would split logic between thunks and reducers. The roughly three-month migration deleted 3,500 lines of boilerplate and was released gradually to production. The team also identified explicit TypeScript typing for dispatch, selector, and store hooks, plus dispatch().unwrap() for awaited calls, as important details, while atypical reducers were deferred to separate projects.
What Being a Staff Developer Means at Shopify
The post defines a staff developer at Shopify as an individual contributor whose scope and seniority can match a manager’s, while emphasizing that the role is a form of leadership rather than simply a more senior senior developer. Technical excellence comes from varied, challenging project experience, including large-scale systems, refactors, technology shifts, mistakes, and learning to choose appropriate solutions under constraints. The role also pairs with a team manager, translating product and business objectives into technical proposals, judging scope and infrastructure needs, finding recurring product problems, and saying no when effort does not match return. Its impact is measured partly through enabling others via reviews, mentorship, sponsorship, and support, leading to an adaptable leader comfortable with ambiguity whose responsibilities evolve with company needs.
How OVO determined the right technology stack for their web-based projects
OVO faced maintainability issues caused by fragmented web technology stacks, including PHP, Vue, React, Nuxt, and Go, alongside outdated documentation. This fragmentation resulted in severe context-switching costs during code reviews and prolonged onboarding times for new engineers. To establish a single primary front-end stack, the team compared React and Vue against maintainability principles encompassing operability, simplicity, and evolvability. Vue was selected because its standardized framework structure and documentation minimized architectural variations across projects. Implementation entailed creating a standardized Vue boilerplate, updating the internal UI library, and scheduling periodic upgrades for legacy React services rather than migrating them entirely.
Lessons From Linguistics: i18n Best Practices for Front-End Developers
Front-end internationalization can fail when code encodes English assumptions about word order, punctuation, pluralization, gender, or text layout. The post uses an image-status example to show why a complete sentence should be translated as a whole, then examines interpolation through a reusable status bar whose file-type and count values vary across languages. It explains that plural categories can include zero, one, two, few, many, and other, with Polish examples showing why numeric rules should not be hardcoded, and that gender can affect articles and adjectives, as in French. Its recommended practices include keeping linguistic decisions in translation strings, avoiding code-level text construction and elaborate styled layouts, and letting i18n libraries select plural forms from quantities; complex promo layouts may need simplification or language-specific images.
Migrating from Role to Attribute-based Access Control
Grab's streaming data platform team migrated the Kafka Control Plane from Role-Based Access Control to Attribute-Based Access Control to eliminate operational bottlenecks and manual permission management. The previous model required defining hundreds of roles, permissions, and group mappings in an internal IAM service, leading to approval delays and stale memberships. Under the new architecture, user attributes sync from the HRMS and token payloads, while resource attributes are tagged upon creation or backfilled to reflect department and team ownership. Open Policy Agent evaluates access requests defined in Rego via middleware by comparing user attributes with resource metadata. This transition eliminated over 200 roles, 200 permissions, and roughly 3,000 unused IAM resources while automating access provisioning for new joiners.
Improving the Developer Experience with the Ruby LSP
The Ruby LSP is a language server intended to improve Ruby’s developer experience by providing editor features through the Language Server Protocol, without typechecking code or requiring type annotations. It targets performance and stability in Shopify’s Core monolith, where it must analyze thousands of files that may themselves contain thousands of lines, and is opinionated toward Ruby on Rails. The article explains LSP’s client-server model, in which editors exchange JSON requests and responses over STDIN and STDOUT, then focuses on nonpositional features such as Folding Range, including document synchronization and parsing. It also outlines future work on parallel request processing, codebase indexing, caching, and a plugin system for framework-specific capabilities such as Rails navigation and model information.
The 25 Percent Rule for Tackling Technical Debt
Technical debt is framed as four categories that need different responses: yearly debt may involve rewrites or gradual replacement, monthly debt requires team-sized planning, weekly debt can become a sprint card, and daily debt is code tidying during ordinary work. The article cautions that some apparent yearly debt reflects growth or changing markets rather than a past mistake, while smaller debt can be deferred only if it is recorded and scheduled. The proposed 25-percent allocation gives 10% to daily improvements, 10% to weekly work, and 5% to planning monthly and yearly debt, with the latter focused on prioritization rather than implementation. Its conclusion is that sustainable remediation depends on team culture: engineers need permission and encouragement to refactor, teams need space to schedule broader work, and leaders need to weigh costs and tradeoffs.
ShopifyQL Notebooks: Simplifying Querying with Commerce Data Models
ShopifyQL Notebooks gives merchants a self-serve way to investigate changing business metrics, understand their causes, and share findings as data stories. It combines the ShopifyQL query language with commerce-specific data models designed for users without SQL expertise, covering domains such as products, orders, and customers. The modeling process starts with business questions, translates them into columns and an appropriate grain, validates the design with mock data and queries, and evaluates upstream data, freshness, and performance. To support large merchants, the implementation uses incremental models and roll-ups, which can reduce build and query times while adding maintenance, pipeline, and monitoring complexity.
The Complex Data Models Behind Shopify's Tax Insights Feature
Shopify Tax’s Tax Insights feature provides merchants with state-level information about sales-tax liability and approaching liability, amid U.S. rules that vary by jurisdiction, product, threshold, and evaluation period. The implementation modified existing data models, added four new ones, and represented changing tax rules with lookup data containing valid_from and valid_to dates, including thresholds, date ranges, inclusions, exclusions, and inclusive or exclusive comparisons. Teams prototyped models in BigQuery SQL, implemented them in PySpark and Spark jobs, and combined merchant monthly taxable-sales aggregates with the rule model to evaluate economic nexus; the architecture moved source data through GCS and a warehouse into a final model. Each run publishes merchant-keyed insights to Google Cloud Bigtable, where the core application retrieves them for the frontend, while warehouse data supports analytics and other processes.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.