---
title: "Architecture"
description: "834 posts about Architecture, summarised, each linking to the original."
---

# Architecture
> 834 posts about Architecture, summarised, each linking to the original.

## Articles

### [Real-Time Collaboration with Replicache and Fly-Replay](https://yomu.fyi/post/real-time-collaboration-with-replicache-and-fly-replay.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 23, 2022
- Type: Problem & solution

Distributed real-time collaboration becomes difficult when a single server coordinating document changes is overloaded and documents must be split across machines without exposing routing complexity to clients. This demo combines Replicache, a custom WebSocket endpoint, Fly Machines, and the Fly-Replay header to route each document’s connection through a shared router to the machine responsible for that document. The server keeps connections by spaceID, sends initial and incremental state, and tracks messages already delivered; the client overrides Replicache’s push and pull handlers to exchange data through the persistent socket. The resulting application has React frontend, Express router, and Replicache/WebSocket backend layers, with each todo list assigned to one backend machine. The author cautions that the approach is demonstrated only in a basic demo and should receive serious production testing and tuning.


### [Implementing Server-Driven UI Architecture on the Shop App](https://yomu.fyi/post/implementing-server-driven-ui-architecture-on-the-shop-app.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 22, 2022
- Type: Explainer

The Shop Store team implemented a server-driven UI architecture for the Shop App’s Store Screen to personalize sections and layouts for different merchants. Previously, the client rendered a fixed layout, limiting customization, experiment timing, and the speed of fixes after weekly app releases. The new design uses template processing, orchestration, data loading, and a shop-server GraphQL layer, with ProductsSection and CollectionsSection types and configurable GridLayout or ShelfLayout options. On the client, ServerDrivenStoreScreen and StoreSectionContainer render the sections returned by the server, while default layouts allow older app versions to handle unfamiliar section or layout types. The architecture enables backend-controlled experiments and section changes without requiring an app release, while giving the product team a system intended to grow with the store experience.


### [Fetching and caching Supabase data in Next.js 13 Server Components](https://yomu.fyi/post/fetching-and-caching-supabase-data-in-next-js-13-server-components.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: jonmeyers\_io
- Published: Nov 17, 2022
- Type: Tutorial

The article demonstrates data fetching and caching with Supabase in Next.js 13’s experimental app directory, focusing on Server Components and Suspense. It creates a posts table, configures a Supabase client, and uses async components to fetch data without manually managing loading states. The examples cover static generation, dynamic routes with generateStaticParams, time-based revalidation, client-side fetching, and a realtime pattern that combines server-rendered initial data with a client subscription to Postgres changes. The article notes that Next.js 13 is stable while the app directory remains in beta, and concludes that separating caching from fetching lets applications change freshness requirements without replacing their data-fetching logic.


### [Query expansion based on user behaviour](https://yomu.fyi/post/query-expansion-based-on-user-behaviour.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Shuailong Liang
- Published: Nov 16, 2022
- Type: Problem & solution

Grab developed a query expansion framework to resolve search issues such as typos, multi-language terms, missing alternatives, and word spacing. The corpus is generated through manual annotation for the top thousand fat-head queries and session-based data mining of user rewrite patterns. Offline pipelines extract adjacent query rewrites occurring within 30 seconds where only the second query generated a click, limiting candidates to three per query for serving efficiency. In production, expanded keywords pass through a query understanding, multi-recall, and result fusion flow with automated pruning of underperforming pairs. Online A/B testing across six Southeast Asian countries showed improvements in click-through and conversion rates, with data-mined expansions outperforming manual annotations alone.


### [Caching Without Marshal Part 2: The Path to MessagePack](https://yomu.fyi/post/caching-without-marshal-part-2-the-path-to-messagepack.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 15, 2022
- Type: Problem & solution

Rails caching relied heavily on Marshal, but cached Ruby-specific objects could break after code changes when old payloads were loaded by code without the former classes. MessagePack provided a generic binary format whose core types exclude Ruby-specific Object and instance-variable encodings, while extension types allowed production-specific serializers such as the compact Date representation. During a roughly six-month migration, the team ran both formats, prefixed MessagePack payloads with a version byte, and used encoding failures to identify unsupported cache values. To scale beyond 128 extension types, they added an Object catchall based on as\_pack and from\_pack, plus Struct and T::Struct modules whose attribute digests detect refactors and turn stale data into cache misses. After logs showed that Marshal was no longer used, the core monolith moved exclusively to MessagePack, with the Paquito gem extracting much of the migration work.


### [Caching Without Marshal Part 1: Marshal from the Inside Out](https://yomu.fyi/post/caching-without-marshal-part-1-marshal-from-the-inside-out.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 11, 2022
- Type: Explainer

Rails caching commonly relies on Ruby’s Marshal serialization, which can encode complex objects but also embeds their class names in cache entries. Shopify describes an incident in which a beta-flag refactor changed classes while old and new code overlapped during deployment; old code then failed on cached instances containing unfamiliar class names and methods, despite passing CI. The article examines Marshal through Ruby’s marshal.c implementation, covering atomic and composite types, instance variables, object references, circularity via TYPE\_LINK, and core-type subclasses via TYPE\_UCLASS. It presents MessagePack as a more compact, stricter, and controllable alternative intended to make caching safer by default, while deferring the cache migration and implementation details to the series’ next part.


### [Authentication in Ionic Angular with Supabase](https://yomu.fyi/post/authentication-in-ionic-angular-with-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: simon\_grimm
- Published: Nov 8, 2022
- Type: Tutorial

The tutorial builds a simple realtime messaging app with Ionic Angular and Supabase, covering account authentication and database protection. It defines Postgres tables for users, groups, and messages, enables Row Level Security with policies for public reads and authenticated operations, and adds a trigger to copy newly registered users into a public users table. Angular services, route guards, and Supabase client logic handle email/password registration and login, session state, protected navigation, and magic links. For native apps, Capacitor URL events extract access and refresh tokens from a custom-scheme callback before establishing the session and navigating to the groups page. The tutorial concludes that Supabase can support Ionic Angular authentication with magic links, while phone authentication through Twilio is another possible option.


### [The Management Poles of Developer Infrastructure Teams](https://yomu.fyi/post/the-management-poles-of-developer-infrastructure-teams.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 28, 2022
- Type: Explainer

Developer infrastructure managers must balance three poles: management support, system and domain expertise, and organization-wide road maps, with the tension intensified by teams that often lack dedicated product managers and support critical systems for hundreds or thousands of users with six to eight developers. Domain-focused teams build deep knowledge needed for maintenance, migrations, incident investigation, on-call coverage, and developer-experience improvements, but their local road maps can diverge from the department’s highest-impact opportunities. The proposed responses are temporary individual assignments, having a whole team contribute to another team’s goals, and forming a cross-functional tiger team; each shifts trade-offs among focus, maintenance, expertise, motivation, and management oversight. A month-long tiger team built the first Spin proof of concept, which led to a dedicated team, but no structure eliminates the tensions, so choices must be judged according to context.


### [Hubble: Our Tool for Encapsulating and Extending Security Tools](https://yomu.fyi/post/hubble-our-tool-for-encapsulating-and-extending-security-tools.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 21, 2022
- Type: Explainer

Shopify’s Trust team faced rising complexity in securing a distributed device fleet, including many tools, provisioning and patching work, varied networks, and the loss of in-person remediation. Hubble was built as a unified layer over mobile device management and other security systems, ingesting and standardizing their data while sending commands back with granular access controls and centralized auditing. IT staff use it for inventory, device management, and security, while employees can view device health and compliance, receive remediation guidance, and manage test-device or beta participation. The platform also serves as a standardized source for automation, and the post presents encapsulation and cross-team investment as ways to reduce administrative overhead, support proactive security, and limit risk.


### [How to Structure Your Data Team for Maximum Influence](https://yomu.fyi/post/how-to-structure-your-data-team-for-maximum-influence.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 20, 2022
- Type: Problem & solution

Embedded data science leaders may support multiple product and business areas, creating competing work streams, specializations, and stakeholder relationships. The post compares swim lanes and stochastic process assignments against principles of efficiency, influence, stakeholder clarity, stability, growth, and flexibility. It recommends a hybrid “diamond defense” in which scientists are loosely assigned to zones for specialization while a bullpen, including the manager and an additional scientist, supplies reserve capacity. Resources can be redirected to overloaded areas, and team members can rotate into vacancies or cover leave. The framework is presented as combining clear ownership and subject-matter depth with the flexibility to handle change, though the post warns that excessive rotation can create volatility and turnover.


### [The Engineering Story Behind Flex Comp](https://yomu.fyi/post/the-engineering-story-behind-flex-comp.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 5, 2022
- Type: Explainer

Shopify’s Flex Comp lets employees allocate total rewards among base salary, Restricted Stock Units (RSUs), and Stock Options, with allocation changes during windows and no one-year equity cliffs. Building the system required a prioritized backlog, cross-functional teams, early API contracts, and privacy controls around its code and data. For sensitive analytics events, the team encrypted entire Kafka payloads with RSA, using 4096-bit keys for ordinary events and 6144-bit keys for larger payloads, while later identifying hybrid encryption as a better design. On the frontend, typed hooks, mocked data, and artificial delays allowed business logic and interface work to proceed in parallel while contracts evolved. Version one shipped to Shopify’s employees, with documented technical and design debt and future decisions retained for subsequent iterations.


### [Postgres WASM by Snaplet and Supabase](https://yomu.fyi/post/postgres-wasm-by-snaplet-and-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: burggraf
- Published: Oct 3, 2022
- Type: Explainer

Supabase and Snaplet open-sourced postgres-wasm, a PostgreSQL server that runs inside a browser and supports browser persistence, pg\_dump restoration, and logical replication with a remote database. Rather than compiling PostgreSQL directly to WebAssembly, the project packages a stripped-down Linux image with PostgreSQL in a Buildroot-based virtual machine, then runs it through v86 in the browser. PostgreSQL 14 startup failures were resolved by disabling v86 JIT compilation, while image optimization used a compressed 9P filesystem, asynchronous HTTPS loading, and page\_poison=on to produce an initial state of about 12MB. Websocket proxying provides outbound connectivity and a forked proxy with nginx tunnels incoming connections to PostgreSQL. The project remains experimental because the download is currently about 30MB, but the post identifies offline data, testing, development environments, snapshots, and support as potential uses.


### [Lessons From Building iOS Widgets](https://yomu.fyi/post/lessons-from-building-ios-widgets.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 22, 2022
- Type: Explainer

Shopify’s widget team describes a third-generation redesign for iOS and Android after the original iOS widgets offered only two metrics and consumed substantial space. The widgets target merchants who need fresh shop analytics, using native SwiftUI on iOS because WidgetKit extensions have memory constraints, serialize SwiftUI view hierarchies to disk, and lacked official React Native support. Independent widget-side fetching uses shared session-management code and tokens; repeated update callbacks caused duplicate network requests and token-refresh races, so the team introduced a short-lived, widget-specific cache. Replacing existing widgets also exposed migration limits: changing widget kind identifiers could blank or remove installed widgets, leading Shopify to deprecate old widgets while releasing new ones and guide merchants through the transition. The team planned further customization and integration with iOS 16 features, including watch and lock-screen complications and live activities.


### [Four Approaches to Debugging Server-side WebAssembly](https://yomu.fyi/post/four-approaches-to-debugging-server-side-webassembly.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 7, 2022
- Type: Explainer

Server-side WebAssembly offers Shopify Functions low cold-start latency and robust security, but debugging remains difficult because step-debugger support is less refined. The post compares four approaches: compiling to a native architecture, using LLDB or GDB with Wasmtime, using a dedicated WebAssembly debugger, and using browser developer tools. Native debugging generally provides the best variable rendering and a familiar workflow, though it may require stubs for external calls and cannot reproduce WebAssembly-specific behavior. Wasmtime keeps execution closer to the target environment, while dedicated and browser-based tools offer WebAssembly-specific capabilities but have missing features or lower-fidelity variable views. The recommended choice depends on whether the issue reproduces natively, and browser-based tooling may become more accessible as it improves.


### [Focus On Behavior, Not State, for a More Maintainable Codebase](https://yomu.fyi/post/focus-on-behavior-not-state-for-a-more-maintainable-codebase.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 1, 2022
- Type: Explainer

Shopify’s work adding pre-orders and “try before you buy” to its Purchase Options APIs exposed code that coupled selling plans too tightly to subscriptions, making future purchase models harder to add. The team reframed the domain around orders and behaviors, treating subscriptions as recurring orders and distinguishing recurring or deferred delivery and billing where those properties mattered. It introduced a wrapper around selling-plan Active Record objects, replaced those objects outside their defining component, and exposed methods such as whether deliveries are recurring while keeping policy details encapsulated. Because platform behavior and merchant reporting need different concepts, the team also added a translation table populated at order creation, with calculated or merchant-provided labels and optional trait records; the maintenance cost isolates reporting semantics from platform and data-team logic.


### [Building a Realtime Trello Board with Supabase and Angular](https://yomu.fyi/post/building-a-realtime-trello-board-with-supabase-and-angular.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: simon\_grimm
- Published: Aug 24, 2022
- Type: Tutorial

The tutorial builds a Trello-style project-management board with Supabase, Angular, and Tailwind rather than stopping at a small “hello world” example. It defines boards, lists, cards, users, and user\_boards in Postgres SQL, adds magic-link authentication, and uses Row Level Security policies plus helper functions and triggers to control access and maintain relationships. Angular components and services use the Supabase JavaScript client, while realtime is enabled for cards and lists and exposed through an Observable that applies INSERT, UPDATE, and DELETE events to local board state. The resulting clone supports shared boards for invited, already registered users, and the article provides SQL and source code for running the project with a Supabase instance.


### [PostgREST v10: EXPLAIN and Improved Relationship Detection](https://yomu.fyi/post/postgrest-v10-explain-and-improved-relationship-detection.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: steve\_chavez
- Published: Aug 19, 2022
- Type: Announcement

PostgREST v10 adds HTTP and supabase-js access to PostgreSQL execution plans for reads, writes, and RPC requests, with the feature restricted to development-oriented use through db-plan-enabled. Plans can be returned as text or JSON, analyzed to show execution time and RLS filtering, and queried in verbose form for the pg\_stat\_statements query identifier and related settings. The release also improves relationship detection: unique or foreign-key primary keys produce one-to-one JSON objects, while computed relationships let SQL functions define efficient links for views and materialized views. Many-to-many detection now requires a join table’s primary key to include both foreign-key columns, a breaking change; the post says PostgREST 10 was not yet on the Supabase Platform and was planned for release there over the following month.


### [Realtime: Multiplayer Edition](https://yomu.fyi/post/realtime-multiplayer-edition.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: wenbo,stas
- Published: Aug 18, 2022
- Type: Announcement

Supabase announces the general availability of Realtime’s multiplayer features, Broadcast and Presence, alongside a new channel-based architecture and supabase-js support. Realtime originally used Postgres logical replication, rather than LISTEN/NOTIFY, to avoid the latter’s 8,000-byte payload limit and the latency of querying each change from the database. In the new design, Channels act as rooms that support ephemeral publisher/subscriber Broadcast messages, shared Presence state, and Postgres change events; Presence uses Phoenix Tracker, a delta-based CRDT, for eventually consistent synchronization and removes disconnected users automatically. Realtime now runs as a distributed Elixir and Phoenix cluster on Fly, with clients connecting to the nearest node and nodes communicating across a backbone network to improve connection speed, stability, and latency. Free-plan active projects and projects created from August 18 have access, while other projects are scheduled for migration over the following weeks.


### [How We Enable Two-Day Delivery in the Shopify Fulfillment Network](https://yomu.fyi/post/how-we-enable-two-day-delivery-in-the-shopify-fulfillment-network.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Aug 18, 2022
- Type: Explainer

Shopify Fulfillment Network (SFN) enables merchants to offer two-day delivery across the United States by balancing inventory across Shopify-certified warehouses and choosing suitable carrier services. Shopify’s platform adds GraphQL APIs that let any third-party logistics provider upload pre-computed delivery dates, while storefront and checkout surfaces display them to buyers. SFN calculates schedules using inventory proximity, product weight, dimensions and hazard constraints, warehouse calendars and capacity, carrier pickups, closures, capacity limits, and time-in-transit data. It groups SKUs with shared inventory locations and characteristics into reusable schedules, maps each SKU to one, and supports direct delivery-date lookups in under 100 milliseconds; the selected date is returned to SFN for fulfillment routing.


### [How we store and process millions of orders daily](https://yomu.fyi/post/how-we-store-and-process-millions-of-orders-daily.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Xi Chen
- Published: Aug 15, 2022
- Type: Problem & solution

The Grab Order Platform processes millions of food and mart transactions daily, requiring high throughput, fault tolerance, and reduced cloud costs across transactional and analytical workloads. To meet these demands, the engineering team decoupled their database architecture by using Amazon DynamoDB for critical OLTP queries and MySQL RDS for historical OLAP queries. DynamoDB handles online order lifecycles with strong consistency, utilizing sparse Global Secondary Indexes for ongoing orders and TTL configurations to limit storage growth. Updates propagate asynchronously to MySQL RDS through a Kafka ingestion pipeline backed by Amazon SQS retries and timestamp-based version checks. This dual-database approach isolated core transaction availability from analytical queries and delivered significant cloud cost savings.


[Newer posts](https://yomu.fyi/topic/architecture/page/32.md) · [Older posts](https://yomu.fyi/topic/architecture/page/34.md)