Loading…
Shopify
Global commerce platform powering millions of businesses with essential infrastructure and innovative engineering solutions.
- 370 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Resilient Import Maps - Better Theme Development and Beyond
Shopify found that JavaScript modules and import maps failed in themes and App Blocks when modules loaded before a map, or when multiple maps collided. These restrictions reflected the requirement that module resolution remain stable, but real pages assembled by themes, apps, Shopify, and extensions could not reliably guarantee ordering or a single map. Working with web-platform contributors, Shopify proposed a reconciliation mechanism with one global import map, a resolved module set, and merging rules that preserve existing resolutions and drop conflicting or unsafe additions. The change reached the HTML specification and shipped in Chromium and WebKit; Chrome supports it from version 133 and Safari from 18.4. Shopify also uses es-modules-shim support for browsers without native support, while Firefox support is hoped for.
Evolution of Product Classification at Shopify: From Categories to Comprehensive Product Understanding
Shopify’s product classification system evolved from a 2018 logistic-regression model using TF-IDF into a product-understanding platform designed to capture categories, attributes, characteristics, metadata, and content-safety signals. A 2020 multimodal image-and-text approach improved classification, but growing product complexity led Shopify to combine Vision Language Models with its Standard Product Taxonomy, spanning more than 10,000 categories and over 1,000 attributes across more than 26 business verticals. The production pipeline uses two-stage category and attribute prediction, with category context shaping the second call, and runs on Kubernetes with NVIDIA GPUs; FP8 quantization, Nvidia Dynamo in-flight batching, and KV caching improve inference efficiency. A multi-LLM annotation system independently evaluates products and uses arbitration when models disagree, helping build training data for consistent predictions. The resulting system processes over 30 million predictions daily, delivers an 85% predicted-category acceptance rate, and doubles hierarchical precision and recall versus the earlier neural network approach.
Mobile Bridge: Making WebViews Feel Native
Shopify’s mobile app has about 600 screens, but building every non-critical screen natively or in React Native was costly and slowed development, while conventional WebViews felt slow, disconnected, and out of place. Mobile Bridge addresses this hybrid-app problem with native iOS and Android modules that preload, authenticate, cache, and reuse WebViews, reducing P75 load time from 6 seconds to 1.4 seconds, including network latency and rendering. Built on @remote-ui/rpc, the framework uses JavaScript, CSS, Polaris overrides, title and action APIs, TransportableView, snapshots, and native modal transitions to preserve state and align web screens with native navigation and appearance. WebViews can also invoke native features such as date pickers, barcode scanning, and Add to Wallet; Shopify says the standalone library is being adopted across other products while critical features remain native or React Native.
Five years of React Native at Shopify
Shopify reviews five years of using React Native (RN) as its mobile strategy, adopted to avoid duplicating features across iOS and Android, improve talent portability, and ship more value. Over that period, all of its apps migrated to RN through team-led, non-uniform plans, while native code remained part of the architecture rather than an alternative to eliminate. The company reports sub-500ms P75 screen loads in the Shopify app, similar performance across its apps, and more than 99.9% crash-free sessions, while noting that performance depends on applying sound patterns. Its experience credits hot reloading and TypeScript with developer productivity and cross-platform staffing, but identifies debugging, RN upgrades, and third-party dependency maintenance as costs. Shopify now emphasizes shared libraries, native expertise for platform-specific work, continued New Architecture adoption, and collaboration with the RN ecosystem.
How We Built the BFCM 2023 Globe
The 2023 BFCM Globe is a real-time visualization of purchases made through Shopify-powered merchants worldwide, with animated arcs, city dots, fireworks, camera movement, atmospheric noise, stars, and a custom material. Performance was a central concern because the earlier approach used a separate mesh and draw call for every arc, causing more than 1,000 simultaneous arcs to slow some mobile devices. The revised arc system uses instancing, interleaved per-arc attributes, and vertex-shader Bézier deformation, allowing tens of thousands of arcs in one draw call and an initial benchmark of up to one million arcs on an M1 laptop. Other effects were similarly consolidated, including billboarded point particles for cities and single-draw-call fireworks, while 2D psrdnoise and a stripped-down shader reduced rendering cost. The result combines richer visuals with substantially greater rendering capacity.
How Shopify improved consumer search intent with real-time ML
Shopify describes how it built near-real-time text and image embedding pipelines to support semantic search, replacing keyword matching with representations that enable similarity-based retrieval. Using Google Cloud’s Dataflow and Apache Beam, the pipelines process roughly 2,500 embeddings per second, preprocess images, run GPU-backed inference, and write outputs to both a data warehouse and an event topic for downstream ingestion. The engineering discussion focuses on trade-offs among memory, throughput, latency, and cost: reducing harness threads cut memory use about 2.6x and allowed a return to cheaper workers, while model sharing reduced throughput. For batching, bursty input produced bundles of one, and stateful batching was rejected because its shuffle increased latency; Dataflow’s per-process model instances instead provided enough parallelism to saturate GPUs.
Tophat: Crafting a Delightful Mobile Developer Experience
Shopify built Tophat to reduce the time and friction involved in manually testing mobile pull requests, where switching branches, stashing changes, and waiting for Xcode or Android Studio builds slowed review. The macOS menu-bar application links GitHub pull requests to iOS and Android artifacts produced by CI and stored in Google Cloud Storage, allowing contributors to install builds on simulators or real devices with one click. Tophat parallelizes tasks such as downloading applications and booting devices, and its Quick Launch feature retrieves builds with up-to-date native code for React Native development. The project also addressed real-device iOS installation and signing challenges, integrated with Shipit Mobile, and provided automated setup guidance. After the latest improvements, daily usage doubled, and Tophat was later made open source.
Mitigating Deadlocks in high concurrency environments
MySQL requires locks before updating existing records, and concurrent upserts that mix new and existing rows can deadlock when gap locks overlap, even when processes target different records. The post describes ETL jobs partitioned by account whose updates nevertheless contend because sequential primary-key ordering places records in shared index gaps. It proposes replacing the existing primary key with a composite key of account and id, aligning InnoDB's physical row ordering with the processes' update partitions; gap locks remain necessary, but are largely compartmentalized within each process. An additional unique index on id preserves standalone record identification and compatibility with existing joins, while application and ETL logic remain unchanged. The reported outcome was removal of artificial delays and significantly higher import throughput through reduced job wait time.
Shipping support for module script integrity in Chrome & Safari
Shopify’s checkout is a managed service requiring control over scripts to support high-performance, PCIv4 DSS-compliant checkout and prevent tampering. SRI could validate top-level resources but did not cover imported JavaScript modules, forcing a shim for dynamic imports that added maintenance and runtime overhead. The work revived Guy Bedford’s proposal to put integrity hashes in import maps, enabling unchanged dynamic imports while also covering static imports, top-level module scripts, and modulepreload. Support shipped in Chromium 127 for Chrome and Edge, and landed in Safari 18’s beta release, with just over two weeks of full-time work and about seven chronological weeks overall.
Improving Shopify App’s Performance
Shopify’s mobile app saw performance decline during its React Native migration, prompting a year-long effort to make critical screens load under 500ms at P75 and launch within 2s at P75. The team built real-time dashboards to filter results by device and operating system, then targeted excessive initial rendering, unnecessary background updates, launch overhead, and weak cache utilization. They introduced LazyScrollView and ListSource on FlashList, deferred work outside the viewport, enabled inlineRequires, optimized a native module, batched state updates, froze background components, and improved Restyle. They also fixed a graph QL cache issue and pre-warmed critical screen data, increasing cache-first delivery to as much as 90% for key screens. By year-end, app launch P75 was 44% faster and screen-load P75 times were reduced by 59%, with performance treated as an ongoing process requiring monitoring and repeated optimization.
A Packwerk Retrospective
Shopify’s retrospective examines Packwerk, a Ruby gem released in September 2020 to enforce package boundaries and help modularize Rails applications. It describes the tool as static analysis that checks constant references against a declared, non-circular dependency graph, with todo files allowing existing violations to be recorded while new ones are blocked. The authors recount how privacy checks shifted attention toward API design, introduced awkward file organization, and were removed in version 3.0. They also explain that Packwerk cannot determine whether package boundaries or violation fixes reflect runtime coupling, so semantic grouping can mislead developers and add harmful indirection. Shopify ultimately treats running code as the strongest progress signal and retains Packwerk mainly to prevent new dependencies at the application’s base layer.
Horizontally scaling the Rails backend of Shop app with Vitess
Shop app’s hockey-stick growth pushed its Ruby on Rails backend and MySQL primary datastore toward capacity, with multi-terabyte storage, week-long schema migrations, and throttled background jobs. After splitting databases and scaling surrounding systems, the team adopted Vitess to shard and re-shard MySQL while retaining SQL compatibility and self-managed primary datastores. They organized user-owned data into a sharded “users” keyspace using user_id, kept other data in an unsharded “global” keyspace, and built application-layer query verifiers to validate routing, distribution, and transaction correctness before migration. The move made schema migrations take hours instead of weeks, removed the reported capacity-driven throttling, and enabled further scaling by adding shards, while introducing concepts such as Sequences, Vindexes, and VSchemas that developers must learn.
Getting Started with React Native Skia
React Native Skia exposes Google’s cross-platform Skia 2D graphics engine to React Native, giving developers a canvas for custom shapes, visual effects, and dynamic interactions. The article explains when to choose it over react-native-svg or standard React Native Views: SVG is optimized for static SVGs, while Skia supplies primitives such as shaders, path operations, and image filters for animated experiences. It introduces the Cartesian canvas, basic Circle, Rect, and RoundedRect shapes, and custom paths created with SVG notation or imperative commands including addCircle, lineTo, and addPoly. Examples then combine Skia with react-native-reanimated shared values for UI-thread animation and react-native-gesture-handler for panning, before adding Blur and DashPathEffect; the article also notes web support through React Native Web and points to further examples.
Introducing Ruvy
Ruvy is an open-source toolchain that takes Ruby code as input and produces a WebAssembly module whose exported _start function executes that code. Built on ruby.wasm, it pre-initializes the Ruby virtual machine and includes Ruby files during the build, while avoiding the need to supply a Ruby script path as a WASI argument at runtime. In Wasmtime benchmarks, Ruvy’s instantiation and _start execution took about 44.4–45.2 ms, compared with 55.8–56.9 ms for ruby.wasm plus wasi-vfs, and its Wasm-to-native compilation took about 440–457 ms versus 1.62–1.68 seconds. These results correspond to roughly 20% faster runtime performance and about 70% less compilation time in the reported tests. Ruvy currently requires users to install build dependencies and compile it because precompiled binaries are not shipped.
Building a ShopifyQL Code Editor
ShopifyQL Notebooks, released in October 2022, is a first-party app that lets merchants analyze shop data. To give its guided code-editing experience, Shopify used CodeMirror, which did not natively support ShopifyQL. Because ShopifyQL’s grammar is defined in ANTLR and generates Go and TypeScript code, while the language server follows Microsoft’s Language Server Protocol (LSP), the existing client and server language infrastructure was retained. A custom adapter passes queries to the language server, converts its incremental five-integer token descriptors into CodeMirror-compatible offsets, and reshapes them into a buffer of Lezer nodes for parse-tree construction. The resulting integration supports syntax highlighting, code completion, linting, and hover tooltips through CodeMirror plugins, while using its internal parse tree to inform editing behavior.
Sidekick’s Improved Streaming Experience
Sidekick addresses two streaming problems in LLM chatbots: Markdown fragments can render awkwardly before syntax is complete, and tool-driven responses can delay visible output across multiple LLM roundtrips. Its solution combines a buffering Markdown parser with an event emitter that multiplexes the main response and asynchronously resolved content into one stream. A Node.js Transform stream processes Unicode characters through a finite-state machine, buffering ambiguous sequences such as emphasis, list markers, and links until they resolve or are flushed as raw text. The parser also recognizes card links, triggering asynchronous card-resolution tasks while Server-Sent Events carry named streams to the UI, which demultiplexes them into the main response and placeholders. The design uses Markdown as a human-readable transport between the LLM, backend, and frontend, allowing content to appear as it is produced.
Shopify’s platform is the Web platform
Shopify now recommends Remix for building Admin apps, which merchants use to configure apps, view data, and integrate services within Shopify Admin. Because these apps run in cross-origin iframes, browser restrictions can block cookies, storage, URL updates, and direct access to the Admin interface; App Bridge uses postMessage() to restore capabilities and support deeper integration. The new App Bridge client replaces the previous package, requires only a script tag, and makes standard web APIs such as history.pushState() work with Admin apps. The Remix template uses file-based routes, server-side loaders and actions, implicit API handling, and Shopify-specific integrations supplied by @shopify/shopify-app-remix, with Prisma provided for storage and sessions. Shopify integrated the template into Shopify CLI and describes it as the beginning of broader platform and design-system reworks.
Contributing support for a Wasm instruction to Winch
Adding a WebAssembly instruction to Winch, Wasmtime’s baseline non-optimizing compiler, requires coordinated changes across code generation, register management, ISA-specific emission, and tests. The contribution described adds visitor support, delegates through CodeGenContext and MacroAssembler, emits x86_64 instructions through Cranelift-backed assembler methods, and registers the instruction for differential fuzzing. CodeGenContext keeps the value stack, register allocator, and function frame consistent, while spilling can move values to linear-memory offsets; MacroAssembler handles ISA differences, register constraints, and CPU feature fallbacks such as lzcnt versus bsr-based clz. Filetests inspect generated machine code without executing it, while differential fuzzing compares Winch results with another WebAssembly engine across randomly generated modules and inputs. The workflow also includes running fuzzing, incorporating draft-PR feedback, rebasing, and submitting the public Wasmtime pull request.
Creating a Flexible Order Routing System with Shopify Functions
Shopify’s smart order routing replaces a one-size-fits-all approach to choosing which inventory location fulfills an order with configurable routing rules. Rules can rank eligible locations by proximity to the destination, staying within the destination market, or merchant-defined location priorities, and merchants can arrange them in an order in the admin. Each rule is a Shopify Function written in Rust, receives data shaped through a GraphQL query, and returns location rankings; a reducer resolves ties by applying later rules to locations tied at rank 0. The rebuilt process uses the pipeline pattern to separate input preparation, rule execution, reduction, and final location selection, improving readability and debugging compared with the earlier tightly coupled implementation. This first stage dogfooded Shopify Functions on its own merchant stores, with a later stage planned to let developers add more complex rules.
Adventures in Garbage Collection: Improving GC Performance in our Massive Monolith
Shopify investigated high Ruby garbage-collection latency in its monolith, where minor marks took about 3 ms but major marks could exceed four seconds and some requests spent multiple seconds in GC. Logging showed that roughly 75% of major marks were triggered by oldmalloc and nearly 25% by shady, while nofree became dominant after oldmalloc was addressed. The team used production experiments, improved metrics, changed GC environment settings, increased initial heap slots, and added out-of-band collection during request processing. A 128 GB oldmalloc threshold reduced major marking by about 20%; later out-of-band collection cut the P99.99 and P99.9 GC-time tail by almost tenfold and largely eliminated median-request pauses. The resulting configuration, with collection frequency adapting from every 128 to every 512 requests, was deployed across the fleet after about three weeks.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.