---
title: "Architecture"
description: "834 posts about Architecture, summarised, each linking to the original."
---

# Architecture
> 834 posts about Architecture, summarised, each linking to the original.

## Articles

### [React Native Skia—For Us, For You, and For Fun](https://yomu.fyi/post/react-native-skia-for-us-for-you-and-for-fun.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 29, 2022
- Type: Announcement

React Native Skia brings Skia’s cross-platform 2D drawing primitives to React Native, with the stated goals of improving its limited graphical capabilities and aligning designers and developers around capabilities comparable to Figma. The integration uses React Native’s JavaScript Interface (JSI), enabling direct JavaScript-to-native communication through C++ rather than asynchronous messages, and exposes an API described as virtually 100% compatible with Flutter’s. On top of the imperative API, a declarative layer uses the React Reconciler to diff internal representation states and send changes to the Skia engine, supporting composable drawings and declarative optimizations. The library includes image filters, shaders, SVG, path operations, vertices, text layouts, mesh gradients, shadows, backdrop blur, and animations, with a direct connector to Reanimated 2; the authors expect it to address a major React Native pain point and describe the work as just beginning.


### [Graph Networks - Striking fraud syndicates in the dark](https://yomu.fyi/post/graph-networks-striking-fraud-syndicates-in-the-dark.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Muqi Li
- Published: Apr 28, 2022
- Type: Problem & solution

Grab's Integrity team faced escalating threats from organized fraud syndicates operating device farms and GPS spoofing tools that mimic normal user behavior. To overcome the limitations of traditional rule-based systems, Grab built a Graph-based Prediction Platform that links entities across shared attributes to expose hidden network structures. The architecture includes a scalable graph database platform holding over one billion nodes, powering real-time graph visualization for analysts alongside a configurable network feature extraction system. In addition, the team deployed Graph Neural Networks to capture structural correlations across transaction networks. These systems collectively help detect coordinated promo abuse, payment fraud, and complex money laundering schemes.


### [Building a Business System Integration and Automation Platform at Shopify](https://yomu.fyi/post/building-a-business-system-integration-and-automation-platform-at-shop.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 22, 2022
- Type: Problem & solution

Shopify needed a maintainable way to connect increasingly modular business systems whose interfaces ranged from SOAP and FTP to REST and GraphQL. The company evaluated Mulesoft but found that its proprietary XML and Java virtual machine stack isolated the team from Shopify’s development ecosystem and complicated access to internal services and infrastructure. It instead built a Ruby on Rails integration platform around minimal validation, transparent data transformations, flexible client support, security, reusable components, documentation, and request-level testing with WebMock. The implementation favors direct manipulation of JSON hashes because that keeps mappings visible, while acknowledging risks such as typos and complex mock management. After a year, the platform had 18 integrations, had migrated all Mulesoft applications, and enabled other teams to build integrations, with ongoing work on testing and onboarding.


### [Möbius: Shopify’s Unified Edge](https://yomu.fyi/post/mo-bius-shopify-s-unified-edge.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 20, 2022
- Type: Problem & solution

Shopify had more than six traffic paths for its core, services, PII-restricted systems, and mTLS-authenticated APIs, making platform-wide features harder to deploy and traffic behavior harder to observe. Möbius and Shopify Front End (SFE) were designed together: Möbius terminates TCP and TLS near clients and forwards requests to SFE, while SFE routes them from regional clusters to application clusters. SFE uses nginx deployments with custom Lua and a centralized API fed by Kubernetes controllers, enabling routing and active/active or active/passive load balancing without DNS TTL delays. Möbius automates edge domain configuration, DNS ownership validation, and certificate acquisition through vendor APIs, so application owners generally only need to change DNS. The unified design standardizes edge protection and observability, supports future edge capabilities, and leaves exceptions for services whose dependencies could create crash loops.


### [Code Ranges: A Deeper Look at Ruby Strings](https://yomu.fyi/post/code-ranges-a-deeper-look-at-ruby-strings.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 14, 2022
- Type: Explainer

Ruby's string model supports mutable and immutable operations, per-object or per-file freezing, and more than 100 encodings, while permitting byte sequences invalid for their attached encoding. This flexibility helps legacy and unusual platforms but makes encoding checks, validity scans, and character-boundary handling part of many operations. To reduce repeated O(n) validation scans, Ruby implementations cache each string's state in a four-valued code range: unknown, 7bit, valid, or broken. The post explains how these values map to public methods, how MRI stores them in two header bits, and how JRuby and TruffleRuby represent them differently. It also describes code ranges as inputs to fast and slow paths and as useful but fallible cache information, especially for native extensions and string-heavy metaprogramming.


### [~Don't~ Repeat Yourself](https://yomu.fyi/post/don-t-repeat-yourself.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Apr 5, 2022
- Type: Explainer

Hugging Face's Transformers library intentionally departs from the traditional "Don't repeat yourself" (DRY) software design principle by adopting a single model file policy. This architecture places all code required for a model's forward pass into a dedicated file, avoiding centralized shared modules for components like attention mechanisms. Because machine learning research evolves rapidly and published model architectures remain static, isolating code protects individual models from breaking changes and lowers contribution barriers for the open-source community. To maintain consistency without manually syncing duplicated code across successor models, the library uses automated tooling driven by Copied from comments. While this approach complicates unified API enforcement and component-wide research integration, Transformers validates API consistency by running approximately 20,000 automated tests daily.


### [Supabase Realtime, with Multiplayer Features](https://yomu.fyi/post/supabase-realtime-with-multiplayer-features.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Wen Bo Xie
- Published: Apr 1, 2022
- Type: Announcement

Supabase announced the third major release of Supabase Realtime, evolving the system from a single-tenant PostgreSQL change listener into a multi-tenant, distributed real-time platform. Built using Elixir and the Phoenix framework across a distributed node cluster, the new architecture decouples the database listener into an extension model and opens generic WebSocket functionality. New multiplayer capabilities include Presence, which uses Conflict-free Replicated Data Types (CRDTs) to track user connection states, and Broadcast, which distributes ephemeral messages across topics without writing to disk. The modular extension design allows listening to multiple PostgreSQL instances, external systems like MySQL, and financial or blockchain feeds, as demonstrated on a globally distributed 20-node cluster.


### [Supporting large campaigns at scale](https://yomu.fyi/post/supporting-large-campaigns-at-scale.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jie Zhang
- Published: Apr 1, 2022
- Type: Problem & solution

Grab developed a batch job service within its Trident automation engine to execute multi-step marketing campaigns for millions of users simultaneously. The system replaces sequential, single-server execution with a distributed architecture powered by Apache Kafka, which distributes batches of 100 users across server clusters using hashed partition keys. To reduce network overhead and queries per second, downstream reward and messaging services introduced batch endpoints backed by bulk database queries, decreasing API latency by up to 85%. Grab further optimized performance by sharding Kafka topics by country and action type to prevent long-running reward tasks from blocking time-sensitive messaging workloads. Additionally, making terminal messaging calls asynchronous allows subsequent batch processing to proceed without waiting for message delivery confirmations.


### [Edge Functions are now available in Supabase](https://yomu.fyi/post/edge-functions-are-now-available-in-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Inian Parameshwaran
- Published: Mar 31, 2022
- Type: Announcement

Supabase launched Edge Functions to execute TypeScript code close to end-users across more than 30 data centers. Built on the secure-by-default Deno runtime and hosted via Deno Deploy, the feature offers fast start-ups, scale-to-zero pricing, and native support for npm modules and Node built-in APIs. Incoming requests first reach an API gateway called Relay, which authenticates JSON Web Tokens, handles rate-limiting, and forwards requests using a Deployment ID to Deno Deploy. Observability is handled through Logflare to feed structured logs and metrics directly into the Supabase Dashboard. To mitigate abuse risks experienced during earlier product rollouts, invocations are currently limited to POST requests rather than general website hosting.


### [Best-in-Class Developer Experience with Vite and Hydrogen](https://yomu.fyi/post/best-in-class-developer-experience-with-vite-and-hydrogen.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 31, 2022
- Type: Explainer

Hydrogen, Shopify’s framework for custom React storefronts, chooses Vite to improve developer experience while targeting performance for end users and evergreen browsers. Traditional bundling with tools such as Webpack can become complex and slow during development because each change may require processing more project files; Vite instead serves user code through native ES Modules, while pre-bundling and caching third-party dependencies. Vite combines an HMR development server with sensible defaults, ESBuild compilation, Rollup’s plugin system, CommonJS-to-ESM conversion, and SSR support, including transformations that differ between browser and server environments. Hydrogen integrates Vite through four plugins covering Hydrogen configuration, React Server Components, SSR and RSC middleware, and React fast refresh, enabling server components, streaming requests, and caching; the text concludes that Vite is a suitable choice, while acknowledging Webpack remains appropriate for some advanced use cases.


### [10 Books Shopify’s Tech Talent Think You Should Read](https://yomu.fyi/post/10-books-shopify-s-tech-talent-think-you-should-read.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 29, 2022
- Type: Explainer

Shopify presents ten books recommended by its developers and engineers for people working in technology, spanning technical foundations, software design, learning habits, feedback, leadership, documentation, and focus. The recommendations were gathered from ten technical employees, while Shopify says its book bar stocks favorite titles and makes requested copies available to employees. Several entries address software systems directly: The Elements of Computing Systems guides readers through hardware, assembly language, programming languages, and operating systems, while A Philosophy of Software Design examines managing complexity and distinguishing tactical from strategic coding. Other selections argue for deep practice, smarter questioning, effective feedback reception, incremental knowledge transmission, decentralized leadership, critical reflection on the technology industry, and uninterrupted work blocks, with the contributors describing how particular books changed their studying, systems thinking, or approach to growth.


### [How telematics helps Grab to improve safety](https://yomu.fyi/post/how-telematics-helps-grab-to-improve-safety.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Wilson Burhan
- Published: Mar 24, 2022
- Type: Problem & solution

Grab developed an in-house telematics engine to monitor driver-partner habits, detect crashes, and improve ride safety beyond passenger feedback. The mobile telematics SDK collects accelerometer, gyroscope, and GPS readings, running on-device processing for immediate accident detection while batching ride data for post-trip safety reports. Because triaxial sensor sampling rates vary independently across diverse devices, the SDK synchronizes data through interpolation to a uniform time grid followed by decimation to an output data rate. Telematics signals are transformed to the frequency domain using Fourier Transform for local compression before backend upload. Grab also combines telemetry with spatial data to identify dangerous road zones and plans to expand on-device event detection across all platform verticals.


### [Using Terraform to Manage Infrastructure](https://yomu.fyi/post/using-terraform-to-manage-infrastructure.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 17, 2022
- Type: Tutorial

Shopify’s Conversations team used Twilio TaskRouter to route voice and chat tasks to agents according to routing rules and skills. As routing needs grew, configuration through Twilio’s GUI became difficult to understand and lacked clear change history, tests, and rollback support. Terraform provided an infrastructure-as-code workflow, backed by a reliable API, a Go client library, and a provider implementing CRUD operations; the team built a provider because Twilio did not have one at the project’s start. Resource definitions included schemas, dependencies, and an importer, while Terraform’s plan and apply commands previewed and executed changes. Integrated with Atlantis and GitHub pull requests, the workflow added plans, reviews, revision history, and easier rollbacks, while enabling business and support teams to submit infrastructure changes themselves.


### [Creating a React Library for Consistent Data Visualization](https://yomu.fyi/post/creating-a-react-library-for-consistent-data-visualization.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 15, 2022
- Type: Problem & solution

Shopify found that teams across its more than 10,000 employees used different visualization tools, producing inconsistent chart styling and differing accessibility and print behavior. The Insights team built Polaris Viz, a React library intended to provide shared, accessible defaults while reducing repeated configuration, and focused this work on light and dark themes plus consumer-defined visual identities. The library replaces numerous visual-style props with a theme prop, stores themes through PolarisVizProvider, and uses createTheme to fill partial definitions with defaults and createThemes to preserve Default and Light themes. Themes can inherit from other themes, while a DataSeries colour can override theme series colors for dataset-specific gradients without creating additional themes. Polaris Viz was subsequently released as an open-source project.


### [Real-time data ingestion in Grab](https://yomu.fyi/post/real-time-data-ingestion-in-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Shuguang Xiang
- Published: Mar 14, 2022
- Type: Problem & solution

Service teams at Grab historically had to dual-write transactional data into databases and Kafka, creating data integrity issues during transaction failures alongside substantial schema maintenance overhead. To overcome these limitations and eliminate burst reads from SQL-based queries, the Caspian team built a real-time ingestion platform synchronising MySQL, Aurora, and DynamoDB directly to Kafka. For MySQL and Aurora, the platform uses Debezium with Kafka Connect on ROW-format binlogs, while DynamoDB changes are captured via DynamoDB streams with auto-scaling AWS Lambda functions. Messages encoded in Protobuf are transported via Kafka and ingested into Amazon S3 using a Golang stream processor. This architecture supports search indexing in Elasticsearch, automated data lake pipelines, cross-region disaster recovery replication, and audit trails.


### [After the Refactor: A Path to Faster Rendering with Liquid-C](https://yomu.fyi/post/after-the-refactor-a-path-to-faster-rendering-with-liquid-c.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 10, 2022
- Type: Explainer

Liquid-C is an extension to the Liquid gem that parses, tokenizes, and renders templates, but some Liquid features, including If and For tags, still render in Liquid rather than Liquid-C. The work explains how Liquid-C compiles templates into bytecode for a customer VM and addresses the difficulty of moving instruction and constant pointers when control flow requires forward or backward jumps. It refactors instructions to store each constant’s index in the constants array, using a hash table to avoid duplicate constants and remove the constant pointer. The change reduced one example constants array from 157 objects to 104, but did not drastically improve theme rendering speed; it slightly increased memory usage and parsing time while preparing for GOTO or JMP operations and later If and For support.


### [Adding the V8 CPU Profiler to v8go](https://yomu.fyi/post/adding-the-v8-cpu-profiler-to-v8go.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 4, 2022
- Type: Benchmark

v8go adds support for V8’s CPU Profiler, allowing Go users to measure JavaScript execution in a V8 isolate and inspect sampled top-down call trees with function and engine details. The API work aimed to stay idiomatic for Go, remain close to V8’s API, support future profiler features, and minimize Cgo overhead. The first design loaded profile data lazily, making separate Cgo calls for profile and node properties; the alternative built the complete graph in C++ before returning it to Go. Benchmarks showed lazy loading was faster for a small tree, averaging about 20 microseconds versus 25 for eager loading, while a larger tree reversed the result at roughly 90 versus 60 microseconds. The comparison demonstrates that Cgo costs depend on workload size and that performance assumptions require benchmarks using representative inputs.


### [Abacus - Issuing points for multiple sources](https://yomu.fyi/post/abacus-issuing-points-for-multiple-sources.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Chandrakanth
- Published: Mar 1, 2022
- Type: Problem & solution

Grab needed a centralised points management architecture to issue loyalty points across a growing catalog of products, membership tiers, and external partner exchanges. To address this, the engineering team built Abacus, an issuance platform designed to process millions of daily transactions with high availability. The system ingests completed transaction streams or API calls, dynamically computes points via configured multipliers, and passes calculations through Amazon Simple Queue Service queues. Once the Point Awarding module updates a persistent ledger, Abacus notifies consumers, emits events to Kafka for downstream consumers, and recalculates rolling point expiration dates.


### [Exposing a Kafka Cluster via a VPC Endpoint Service](https://yomu.fyi/post/exposing-a-kafka-cluster-via-a-vpc-endpoint-service.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Fabrice Harbulot
- Published: Feb 18, 2022
- Type: Problem & solution

To replace VPC peering and reduce attack surfaces, Grab exposed a multi-Availability Zone Apache Kafka cluster in its main AWS VPC to clients in a separate GrabKios VPC using AWS VPC Endpoint Service. Because Kafka requires clients to establish deterministic connections to individual brokers, the team configured a Network Load Balancer with unique TCP ports and dedicated target groups for each broker alongside a shared bootstrap port. They added custom listeners on the Kafka brokers to advertise endpoints using private Route 53 CNAMEs rather than raw interface hostnames. To eliminate unnecessary cross-AZ network latency and data transfer costs, the architecture was refined to advertise AZ-specific private CNAMEs mapped directly to zonal endpoint interfaces.


### [Changing a polymorphic\_type in Rails](https://yomu.fyi/post/changing-a-polymorphic-type-in-rails.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Feb 18, 2022
- Type: Explainer

Rails normally persists a polymorphic association’s type as the parent class name, so moving Car under Garage changes stored values from “Car” to “Garage::Car” and can break lookups for older Key records. The post presents an alternative for the Vehicle/Key example: override polymorphic\_name so Garage::Car and Garage::Boat resolve to stable arbitrary strings such as “car” and “boat,” reducing coupling between namespaces and database data. During transition, the association scope must explicitly accept both legacy class-name values and new arbitrary values, because Rails uses only the single value returned by polymorphic\_name when querying; existing rows can then be cleaned up with a migration, script, or MaintenanceTasks. After cleanup, the temporary unscope can be removed, while class and module renames require updating the mappings rather than database records. The approach adds complexity and is presented as worthwhile only when near-term naming changes are likely.


[Newer posts](https://yomu.fyi/topic/architecture/page/34.md) · [Older posts](https://yomu.fyi/topic/architecture/page/36.md)