API Versioning as a Translation Layer
API versioning as an architecture: requests enter through a version header, pass through a chain of reversible version changes, and reach controllers that only ever see the latest model.
Loading…
Topic
Architecture
830 posts about Architecture. Every summary links to the original.
- 830 posts
- 50 companies
- 31 explainers
See the idea
Architecture explainers
BIAN as a Contract Layer: Modernizing Core Banking One Domain at a Time
BIAN gives banks a shared vocabulary and service-domain contracts that sit between business concepts and the systems implementing them. See how domains, contracts, and governed access stack into one architecture.
Argo CD: The Reconciliation Loop Between Git and Your Cluster
Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
5 Ways to Improve Your React Native Styling Workflow
Shop's React Native app began as Arrive's codebase, and its visual overhaul prompted a broader rethink of styling practices. The resulting open-source Restyle library organizes styling around a design system covering spacing, semantic colors, and typography, represented by a theme object that components receive through React's Context API. Reusable Box and Text components map props such as spacing, background color, and text variants to theme values, while responsive properties target predefined screen-size breakpoints and TypeScript restricts props to valid theme options with editor autocomplete. The stated result is a more enjoyable and efficient workflow: Restyle supported the transition from Arrive to Shop, enabled theme switching, helped tailor screens to device sizes, and made dark mode easy to add.
Securing and Managing Multi-cloud Presto Clusters with Grab’s DataGateway
Grab's data engineering team needed to manage data access across growing Presto workloads spanning hundreds of users, thousands of tables, and multiple clouds without exposing infrastructure churn to clients. Out-of-the-box Presto lacked fine-grained schema- and table-level access controls, unified endpoint routing, and seamless cluster switching. To solve this, the team built DataGateway, a smart HTTP proxy and abstraction layer deployed as microservices on Kubernetes. The platform parses incoming SQL statements, verifies user access control lists against target schemas and tables, and dynamically routes queries to appropriate AWS or Azure Presto clusters without requiring endpoint updates. This architecture enabled zero-disruption cluster migrations across 40 clusters and unified access management across a multi-cloud data lake environment.
Supabase Auth
Following an early community launch on Hacker News, the open-source Firebase alternative Supabase faced heavy developer demand for a dedicated authentication and authorization solution. In response, the team evaluated existing open-source tools such as KeyCloak and Kratos before landing on an architecture combining a forked Netlify GoTrue server with PostgreSQL and PostgREST. The resulting Supabase Auth system provides client methods like signUp and signIn alongside authorization powered by PostgreSQL Row Level Security and policies. By leveraging SQL policies to enforce access rules directly in the database, developers avoid repetitive middleware filtering while maintaining granular access controls. Future roadmap plans include adding third-party OAuth providers, custom email templates, and simplified policy management interfaces for non-technical users.
An Introduction to DNS Traffic Management
DNS traffic management applies data-driven DNS responses to direct clients to different service endpoints, extending DNS’s basic role of mapping domain names to IP addresses. The post introduces DNS through its history from HOSTS.TXT, explains zones, records, nameservers, root servers, TTL, and negative TTL, then connects caching behavior to routing decisions. Low TTLs can make changes visible sooner, but repeated DNS resolution adds latency: with a one-second resolution time, a 15-second TTL could add 5,760 seconds per day, compared with 1,440 seconds at 60 seconds; resolver behavior can also make delays slightly longer. Use cases include active-passive failover, active-active traffic sharing, progressive green-blue deployments, and location-based routing, while Shopify’s system evolved into an automated, self-served platform serving more than 40 domains across 12 teams and over 100M requests per 24 hours.
How We’re Solving Data Discovery Challenges at Shopify
Shopify describes data discovery as a response to growing numbers of data assets and the difficulty of finding, governing, and using them across the data lifecycle. At Shopify, pre-Artifact discovery relied on people, Slack, GitHub, and job logs, while teams lacked granular ownership, change-impact visibility, and documented reporting context. They built Artifact, a search-and-browse tool backed by a centralized metadata model, ingestion jobs, Elasticsearch indexes, GraphQL APIs, and a graph database for filterable upstream and downstream lineage. Its landing page supports browsing and plain-English search, while details pages expose documentation, schema, descriptions, usage, ownership, and lineage; initial scope prioritized frequently used, high-impact, queryable assets and downstream reports and dashboards. Since launch in early 2020, weekly Data team usage exceeded 30% and monthly retention exceeded 50%, above expectations, with future work planned for additional assets, integrations, stewardship, change management, notifications, and metadata APIs.
The Journey of Deploying Apache Airflow at Grab
Engineering and data teams across Grab originally operated independent Apache Airflow instances, causing duplicate maintenance overhead and frequent job failures around scaling, logging, and dependency management. To resolve this, a dedicated team developed a centralized orchestration platform that runs isolated, containerized Airflow instances per team on Amazon EKS. The platform categorizes deployments into three size tiers and provisions dedicated Redis brokers, RDS metadata stores, and Vault secret sidecars using Terraform and custom Helm charts. Teams customize container images using shared GitLab CI/CD templates, while worker scaling is handled via Kubernetes Horizontal Pod Autoscalers. Today, the platform runs roughly 20 Airflow instances executing between 1,000 and 60,000 daily jobs per instance.
Media at Scale: Callbacks vs pipelines
Shopify’s Rails monolith needed to add native product videos and 3D models while continuing to support an image infrastructure containing more than 7 billion images. The design question was whether media creation should rely on Active Record callbacks or an explicit pipeline, with transactions protecting interdependent database writes. Callbacks were quick for simple cases, but adding media-specific behavior such as video thumbnails spread conditionals across models and made lifecycle ordering difficult to follow and debug. The pipeline design routes requests through a single Product Create Media Service and media handlers, each organized into before_transaction, during_transaction, and after_transaction steps, while confining logic to one media type. This structure separates concerns, controls creation order, limits model access, and makes implementation details easier to understand and maintain as the feature grows.
The Reformer - Pushing the limits of language modeling
Standard transformer models hit memory bottlenecks on long sequence modeling tasks due to the quadratic asymptotic memory complexity of global self-attention and oversized positional embedding matrices. The Reformer architecture overcomes these constraints to train sequences of up to half a million tokens using under 8GB of RAM. It re-engineers transformer operations using local and Locality Sensitive Hashing self-attention, chunked feed forward layers, reversible residual layers, and axial positional encodings. In empirical benchmarks using google/reformer-crime-and-punishment, axial positional encodings reduce the model parameter count from over 136 million to approximately 2.58 million by factorizing the positional dimensions. This architectural change cuts inference memory consumption from 959 MB down to 447 MB for evaluated benchmark workloads.
How We Built Our In-house Chat Platform for the Web
Grab extended its in-house chat platform to the web to support its internal Customer Support portal. Because the existing TCP gateway only supports unicast connections with one active connection per user, opening multiple browser tabs would repeatedly disconnect previous tabs. Rather than undertaking a complex migration to multicast connections on the server, the team adopted a hybrid client-side strategy using SharedWorker and BroadcastChannel APIs. The implementation uses a SharedWorker to maintain a single WebSocket connection per domain while a BroadcastChannel syncs events across all open tabs. A custom wrapper over the worker manages version transitions during deployments to avoid race conditions across tabs.
Writing Better, Type-safe Code with Sorbet
Ruby repositories can hide unsafe method calls and ambiguous data shapes when types are inferred from variable names alone. Sorbet static type checking addresses these risks with method signatures, typed structs, enums, and interfaces. It can flag a potentially nil return before a chained call, distinguish database output fields from input fields, and enforce contracts for synchronous and asynchronous indexers in a dependency-injected hexagonal structure. The article also describes Sorbet’s gradual typing, including five strictness levels and namespace-by-namespace adoption using the minimum typed level of true. The stated conclusion is that enforced type safety catches errors unit tests may miss and helps prevent unsafe code from reaching production.
Understanding Programs Using Graphs
TruffleRuby uses a sea-of-nodes graph as an intermediate representation after parsing, allowing its just-in-time compiler to optimize Ruby programs and translate them to machine code. The explanation contrasts this graph with an abstract syntax tree, then shows how control flow, data flow, side effects, pure computations, loops, and phi nodes are represented through boxes and arrows. A three-way conditional demonstrates global value numbering: a repeated multiplication becomes one movable computation that can float across branches without changing program behavior. A loop example shows backward control flow and repeated functional computation, while the discussion weighs graph-based optimization against poor compactness and readability at larger scales. Shopify is building graph-drawing and Ruby decompilation tools to inspect optimization at codebase scale.
Evolving Alongside your Tech Stack
Rapid advances in software have expanded both what applications can do and what developers must understand, from efficient code to integration, security, and user experience. As tools make polished interfaces easier to build, engineers are expected to understand users, design sensibilities, system behavior, and constraints such as memory management and CPU load. The post recommends maintaining broad awareness of changing technologies while prioritizing learning around problems actually encountered; Postgres performance issues may warrant understanding traffic and write effects without exhaustive tuning. It cautions against adopting every new framework or database on reputation alone, using MemSQL as an example, and concludes that developers should evaluate trade-offs carefully, build expertise in the tools they operate, and focus on shipping meaningful product work.
Refactoring Legacy Code with the Strangler Fig Pattern
Shopify’s Ruby on Rails Shop model had grown beyond 3,000 lines, accumulated numerous responsibilities, and developed weak semantic boundaries, making store settings a target for extraction. The team used Flog’s ABC-based score—assignments, branches, and calls—to identify a particularly disordered area, then applied Martin Fowler’s Strangler Fig Pattern to move the locked_settings behavior into Capital::SettingsToLock and a dedicated record/data source. The seven-step process defined and tested a new interface, redirected callers, introduced and backfilled the new data source, switched reads, and finally stopped legacy writes and removed the old column and code. Keeping the old system available during the transition supported incremental changes, monitoring, reversibility, and the stated no-downtime requirement; the completed example left the new system in place with legacy remnants removed.
Creating Native Components That Accept React Native Subviews
React Native’s standard native-module documentation does not explain how to reuse an existing native container while rendering React Native views inside it. This Android-focused Kotlin guide builds a native bottom-sheet component with a ViewGroupManager, inflating a CoordinatorLayout XML layout whose second child becomes the draggable sheet content. It overrides addView to place the first React Native child in the main container and subsequent content in the bottom sheet, then exposes the component through TypeScript with style and children props. A sheetState prop accepts collapsed or expanded, while BottomSheetBehaviour emits a BottomSheetStateChange event so native gestures update React state. The guide concludes that ViewGroupManager enables flexible React Native layouts around native functionality, while noting that platform parity remains important.
How We Prevented App Performance Degradation from Sudden Ride Demand Spikes
Grab experienced severe system strain when sudden localized spikes in ride demand, triggered by events like heavy rain or concert dismissals, coincided with driver shortages. These localized bursts overloaded the platform and degraded the experience for users outside the affected areas. To mitigate this, engineers created the Spampede filter, a circuit-breaker mechanism placed at the start of the booking pipeline. The filter converts pickup locations into Geohash Integer buckets and partitions time using Unix timestamps, tracking unfulfilled requests in Redis with atomic increments and time-to-live expirations. When unallocated requests exceed configured thresholds within a specific bucket, the system immediately short-circuits new incoming bookings to protect overall platform stability.
Plumbing At Scale
Grab's backend services process terabytes of data ingress per hour, generating recurring needs for stream transformations, joins, and time-windowed aggregations across diverse workloads. To support these asynchronous processing patterns across their Go ecosystem, the Coban team developed a managed, NoOps event sourcing and stream processing platform. The architecture packages stateless processing pipelines as Kubernetes deployments on AWS, polling Kafka event logs and using ScyllaDB as a shared metastore for stateful needs like deduplication and windowing. Stream processing pods combine ingestion triggers, a worker pool runtime, and user-provided domain logic plugins with customizable failure handling. This infrastructure scales to handle over 300 billion events weekly while maintaining workload isolation and elastic autoscaling.
How We Implemented Domain-Driven Development in Golang
Building GrabPlatform's partner integration self-service portal initially resulted in an unstructured codebase where individual files exceeded 500 lines and lacked proper segregation. Modifying existing functions carried high risks of breaking functionality across imported source collections. To resolve this, the team restructured the Go application using Domain-Driven Design principles in coordination with product domain experts. They mapped business rules into bounded contexts, identified entities and aggregate roots, introduced repository interfaces, and utilized domain events for cross-context communication. The refactoring distributed core functionality evenly, simplified onboarding, and aligned technical terminology with business concepts.
Griffin, an Anti-fraud Risk Rule Engine Making Billions of Predictions Daily
Grab's Trust/Identity/Safety team built Griffin, an in-house anti-fraud risk rule engine designed to process billions of daily predictions across multiple business verticals. Initially, Grab managed fraud rules directly within backend service code, but escalating rule complexity, tight rule interdependencies, and translation gaps between data scientists and developers caused deployment delays and misfiring errors. To overcome the limitations and steep learning curves of third-party engines like Drools, the team separated the workflow into data orchestration and rule-based prediction. Griffin enables analysts and data scientists to author Python-based rules directly via a web portal and reload updated logic into memory without manual developer intervention. By eliminating I/O during rule evaluation and leveraging Gunicorn multi-processing, the engine handles over 100,000 queries per second at peak on six EC2 instances with single-prediction latencies under six milliseconds.
Using Grab’s Trust Counter Service to Detect Fraud Successfully
Grab's Trust Platform team built the Counter service to detect fraud across business verticals like transportation, food, and payments. The platform replaces manual, multi-week engineering cycles with a self-service UI where data analysts can define and experiment with counters independently. Operating on an asynchronous ingestion and synchronous transaction model, the architecture evaluates incoming stream data, enriches it via internal services, and persists aggregated signals to ScyllaDB through Grab-Stats. A multi-bucket strategy partitions queries into fifteen-minute, hourly, and daily granularities to maintain low-latency query aggregations across wide time ranges under strict SLAs.
Five Common Data Stores and When to Use Them
The article compares five common data stores—relational databases, NoSQL databases, key-value stores, full-text search engines, and message queues—and connects each to appropriate design needs. Relational databases use schema-bound tables, primary and foreign keys, SQL, and ACID transactions for durable business-critical data, while NoSQL databases trade strict schemas for flexibility with document and wide-column models. In-memory stores such as Redis and Memcached provide fast access and configurable eviction for caching; Redis additionally supports richer data structures and disk persistence. Full-text search engines target word and substring searches, whereas Kafka uses a distributed append-only log and producer-consumer model to move data reliably between services. The article concludes that the right choice depends on whether data must be permanent, flexible, quickly accessed, searched, or transferred.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.