Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,318 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
Lessons From Building Android Widgets
Shopify revisited its Android widgets alongside an iOS widget effort, aiming to give merchants faster access to frequently changing business metrics. The team chose native Android development because React Native lacked official RemoteViews support, while widget configuration data was persisted with shared preferences and used to build requests. Because Android could invoke onUpdate() multiple times per cycle, they added a one-minute, short-lived cache keyed by widgetID and mindful of configuration such as locale. This reduced redundant network calls, system load, and incorrect analytics, while the widget rollout also exposed migration limitations between old fixed-size widgets and new space-responsive ones, leading Shopify to deprecate the old widgets rather than transform them.
What is a Full Stack Data Scientist?
Shopify defines full stack data science as an ownership-oriented, end-to-end approach to solving business problems across the data science lifecycle. Rather than specializing in one stage, practitioners participate in discovery and analysis, acquisition, data modeling, pipeline development, and production delivery. The role combines business acumen, communication, Python and SQL programming, exploratory data analysis, data engineering, and machine learning, while treating machine learning as a tool rather than a default and starting simple before adding complexity. Shopify describes the model as generalist with T-shaped development: broad capability is paired with deeper expertise in a specific area, enabling individual strengths within teams to address complex challenges.
Choosing a Postgres Primary Key
Choosing a Postgres primary key involves balancing simplicity, storage, sortability, standards compliance, information leakage, and collision avoidance. The discussion starts with natural and surrogate keys, then compares integer and biginteger identity columns with serial sequences, noting that sequences can have gaps and exposed numeric IDs may reveal information. It then examines pseudorandom and secure random numeric IDs before turning to UUIDs, including UUIDv4 and time-based alternatives, whose generation methods and index behavior differ. Postgres integration, size queries, and the pg_idkit library provide practical ways to generate and evaluate identifiers; the suggested rules of thumb favor integers for simplicity, UUIDv4 for collision avoidance, xid for time-based sort friendliness, and UUIDv6 or UUIDv7 for a more standards-oriented choice.
How the SQLite Virtual Machine Works
SQLite processes SQL through a parse-optimize-execute pipeline that turns query text into executable virtual-machine instructions. It tokenizes characters, parses them into an AST such as a Select struct, and uses table statistics to choose between a full table scan and index-based access. The source describes sqlite_stat1's coarse index and duplicate-value counts alongside sqlite_stat4's distributed samples, then traces bytecode that opens the persons table, compares favorite_color with "blue", and returns matching columns. Registers hold intermediate values, cursors iterate through tables, and sqlite3_step() resumes execution after each returned row; the VM has 186 opcodes as of writing, while bind parameters allow prepared statements to skip repeated parsing and optimization.
Four Approaches to Debugging Server-side WebAssembly
Server-side WebAssembly offers Shopify Functions low cold-start latency and robust security, but debugging remains difficult because step-debugger support is less refined. The post compares four approaches: compiling to a native architecture, using LLDB or GDB with Wasmtime, using a dedicated WebAssembly debugger, and using browser developer tools. Native debugging generally provides the best variable rendering and a familiar workflow, though it may require stubs for external calls and cannot reproduce WebAssembly-specific behavior. Wasmtime keeps execution closer to the target environment, while dedicated and browser-based tools offer WebAssembly-specific capabilities but have missing features or lower-fidelity variable views. The recommended choice depends on whether the issue reproduces natively, and browser-based tooling may become more accessible as it improves.
Focus On Behavior, Not State, for a More Maintainable Codebase
Shopify’s work adding pre-orders and “try before you buy” to its Purchase Options APIs exposed code that coupled selling plans too tightly to subscriptions, making future purchase models harder to add. The team reframed the domain around orders and behaviors, treating subscriptions as recurring orders and distinguishing recurring or deferred delivery and billing where those properties mattered. It introduced a wrapper around selling-plan Active Record objects, replaced those objects outside their defining component, and exposed methods such as whether deliveries are recurring while keeping policy details encapsulated. Because platform behavior and merchant reporting need different concepts, the team also added a translation table populated at order creation, with calculated or merchant-provided labels and optional trait records; the maintenance cost isolates reporting semantics from platform and data-team logic.
When Culture and Code Reviews Collide, Communication is Key
The post examines how cultural differences in politeness, directness, and face-saving can distort written code reviews, causing useful feedback to be softened, misread, or ignored. It contrasts a passive-aggressive Shopify pull-request exchange about missing permalink tests with a more curious and explicit version that states why a separate test would be worthwhile. The discussion distinguishes assuming positive intent from asserting it: context can reduce bias, but it must not erase the impact of embarrassing or harmful communication. It recommends treating reviews as collaborative invitations, asking questions, explaining reasoning, acknowledging reviewers as contributors, and looking past stylistic language to shared goals. The conclusion is that clearer communication and shared ownership can make disagreement productive and improve both learning and the codebase.
Shopify Data’s Guide To Opportunity Sizing
Opportunity sizing gives businesses a structured way to estimate an initiative’s potential impact and compare it with the cost of pursuing alternatives. The framework recommends expressing an estimate as an expected change in a metric, with a range, timeframe, and documented assumptions; an annualized view makes initiatives easier to compare. Directional t-shirt sizing uses rough benchmarks for early ideation and sanity checks, while bottom-up sizing uses observed performance from a comparable product or system, such as email click-through rates, to produce a more accurate estimate. Top-down sizing is intended for new initiatives without comparable systems and starts with broader market or audience information before narrowing the estimate. Shopify recommends connecting initiative-level changes to wider business goals, testing estimates against actual results after launch, and refining the process over time.
How SQLite Scales Read Concurrency
SQLite’s write-ahead log (WAL) is presented as the mechanism that overturns the common claim that SQLite is inherently single-user or unable to scale reads. Unlike the rollback journal, which blocks readers and writers from operating at the same time, WAL leaves original database pages in place and appends changed pages to a -wal file, allowing point-in-time snapshots to coexist with ongoing writes. SQLite tracks those entries in a shared-memory (SHM) index: 32KB blocks contain page-number lists and 8,192-slot hash maps, with slot positions calculated as (pgno * 383)%8192 while transaction position determines visibility. The post concludes that WAL mode benefits most applications, while noting that rollback journaling can remain preferable for edge cases such as multi-database transactions.
How We Built Oxygen: Hydrogen’s Counterpart for Hosting Custom Storefronts
Oxygen is Shopify’s hosting counterpart for Hydrogen custom storefronts, designed to provide globally available, fast storefronts with secure Shopify integration without additional third-party hosting costs. Its architecture combines GitHub and GitHub Actions with Cloudflare Workers and edge distribution, Shopify CDN, Admin API, Identity and Access Management, and Oxygen-scoped developer tooling and observability. Requests pass through a Gateway Worker, which validates authorization for the shop and storefront version before using Cloudflare’s Dynamic Dispatch API to route traffic to a Storefront Worker running Hydrogen code. Rather than build proprietary infrastructure, Shopify reused Cloudflare’s worker platform to reduce upfront complexity, scale, and security risks, while its observability stack uses Grafana, Cortex, Loki, and Tempo. Oxygen is used in production by Allbirds, Shopify Supply, Shopify Hardware, and Denim Tears, and is generally available to Plus merchants.
How We Enable Two-Day Delivery in the Shopify Fulfillment Network
Shopify Fulfillment Network (SFN) enables merchants to offer two-day delivery across the United States by balancing inventory across Shopify-certified warehouses and choosing suitable carrier services. Shopify’s platform adds GraphQL APIs that let any third-party logistics provider upload pre-computed delivery dates, while storefront and checkout surfaces display them to buyers. SFN calculates schedules using inventory proximity, product weight, dimensions and hazard constraints, warehouse calendars and capacity, carrier pickups, closures, capacity limits, and time-in-transit data. It groups SKUs with shared inventory locations and characteristics into reusable schedules, maps each SKU to one, and supports direct delivery-date lookups in under 100 milliseconds; the selected date is returned to SFN for fulfillment routing.
Supabase is SOC2 compliant
Supabase reports achieving SOC2 Type 1 compliance and later updates the announcement to say it is SOC2 Type 2 compliant. The post explains SOC2’s five Trust Services Criteria and distinguishes Type 1’s point-in-time audit from Type 2’s observation period, while framing customer expectations and data stewardship as the main reasons for pursuing certification. Supabase used Vanta to monitor controls and collect evidence, selected an auditor familiar with SaaS companies and Vanta, and formalized practices covering device security, access control, policies, logging, monitoring, vendor reviews, and repository protection. The process still required substantial manual evidence, and the company found that audit scope and auditor expectations mattered more than achieving a perfect automated-tool score. Supabase also launched a security center and identified HIPAA certification as a next step.
When is JIT Faster Than A Compiler?
JIT compilation can outperform ahead-of-time compilation or interpretation for highly dynamic languages such as Ruby by optimizing runtime behavior under assumptions that static approaches cannot safely make. Ruby permits redefining operators, equality, iteration, hashing, and other behavior at any time, even from a background thread, forcing an interpreter to check whether definitions changed and limiting AOT code to conservative behavior. YJIT addresses this by compiling frequently used methods with assumptions about current definitions, then invalidating and discarding compiled versions when those assumptions become false; later calls can run interpreted code until a new version is compiled. The post presents this speculative optimization and de-optimization model as the reason a JIT can be faster, while noting that YJIT had known bugs in Ruby 3.1 and was expected to be closer to production-ready for some uses in Ruby 3.2.
How SQLite helps you do ACID
SQLite’s rollback journal provides atomicity, isolation, and durability by preserving database state before a transaction changes the main file. Without journaling, an interrupted 4KB page write could corrupt the database; the journal instead records original pages, then permits recovery if commit steps fail. A journal header stores a nonce, the database’s pre-transaction page count, sector size, and page size, while each page record includes its page number, a copy of prior contents, and a 32-bit checksum. The mode allows concurrent readers but makes writers wait for readers and blocks new readers, using SHARED, RESERVED, and PENDING filesystem locks. Its simplicity brings a concurrency bottleneck; WAL and wal2 provide better support for concurrent readers, which the post says will be covered next.
Mastering React’s Stable Values
React stable values are values, often returned by hooks, that retain the same value across renders, and the post explains why their identity matters in effects, callbacks, memoization, and component rendering. useState and useReducer update functions and useRef objects are stable, while array literals, object literals, and anonymous functions are recreated on each render; useCallback and useMemo can produce stable values under appropriate dependencies. Unstable dependencies prevent useEffect behavior from remaining limited to intended changes and stop useCallback and useMemo cached results from being reused, while an unstable callback passed to useFocusEffect causes repeated unsubscribe and resubscribe cycles. The post also contrasts invoking an inline helper with rendering it as a component, explains why the latter can rebuild a React Native subtree on every render, and recommends defining components outside other components and using React.memo or callbacks judiciously.
10 Tips for Building Resilient Payment Systems
The post distills lessons from five years working on Shopify’s payment infrastructure into guidance for building systems that tolerate failure and large-scale traffic. It frames the advice partly around onboarding developers who were new to payments or Shopify’s scale, and notes that load testing for BFCM 2021 began in July with globally distributed load generation. It recommends low timeouts across HTTP clients and data stores, circuit breakers such as Semian, and capacity reasoning with Little’s Law, rate limiting, and load shedding. Monitoring should cover latency, traffic, errors, and saturation, while structured, machine-readable logs make centralized records searchable; load tests use benchmark gateways that mimic production partner latency and capacity. For incidents, Shopify uses defined roles, Slack-based coordination, and retrospectives that turn incorrect assumptions into safeguards and training material.
SQLite Internals: Pages & B-trees
SQLite stores table rows as encoded byte arrays, and understanding that representation explains how a simple database preserves compact records and supports efficient growth. Using a sandwich table as an example, the post decodes a row’s varints, type values, UTF-8 text, IEEE-754 floating-point value, and integer count, then examines 4KB pages, cell-pointer indexes, and page headers. SQLite’s b+tree places records in leaf pages and key-range indexes in interior pages; page splits add levels as the database grows, while searches traverse only one page per tree level. The examples estimate that a depth-four tree can hold about 500³ leaf pages, or roughly 476GB, while requiring four page reads for a lookup. The discussion concludes that integer storage is more space-efficient than floating-point storage and that internals such as record encoding and PRAGMAs become less opaque when understood.
How LiveView Took on a Life of Its Own
Phoenix LiveView began as an attempt to build dynamic server-rendered applications without writing JavaScript, using the server for interactions such as realtime reservation updates. Its initial design used stateful Phoenix Channels, a WebSocket connection, client event bindings, and full template re-renders sent back as HTML, a model that worked but produced redundant server work and large payloads. Adding Phoenix PubSub let LiveViews subscribe to reservation events and update all connected browsers through ordinary Elixir callbacks, without client changes. The implementation then evolved around diffing dynamic values, shared static template references, and cached live component templates, reducing transferred data and avoiding growing client bundles. The post concludes that this programming model sidesteps HTTP for dynamic features, with compact payloads and less client-side code.
SOC2: The Screenshots Will Continue Until Security Improves
Fly.io’s SOC2 certification prompts an explanation of what the audit measures, when companies should pursue it, and what security value customers should assign to it. The post characterizes SOC2 as an accounting-style audit of documented company security processes, centered on an auditor-customized Document Request List (DRL) and screenshots, rather than adversarial product testing. It distinguishes Type I’s point-in-time snapshot from Type II’s multi-month confirmation and recommends doing the engineering work early, while delaying certification until customer demand justifies it. Fly.io prepared for more than a year and describes controls including Google SSO, certificate-based SSH access through Teleport, VPN gating, GitHub SSO, and session audit logs. The resulting certification improved processes but remains a weak positive security signal: SOC2 can miss much of a company’s technical security model, so penetration tests and clear security reporting still matter.
Shopify’s Technical Interview Process: What to Expect and How to Prepare
Shopify’s guide outlines the interview process for individual-contributor technical roles, while noting that stages can vary by role and team and that leadership hiring may add team-specific interviews. Candidates may have a one-hour recruiter “Life Story” conversation, a 40-minute CoderPad coding exercise, a 75–90-minute remote pair-programming session, and a 60-minute Technical Deep Dive focused on previous projects. The coding exercise assesses algorithmic thinking and problem solving, while pairing emphasizes communication, implementation, edge cases, scalability, limitations, and testability. The deep dive asks candidates to discuss impact, architecture, technical decisions, trade-offs, failures, successes, lessons learned, teamwork, and mentorship; the guide also recommends clarifying questions, running code early, using permitted references, and choosing projects they know well.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.