Loading…
Shopify
Global commerce platform powering millions of businesses with essential infrastructure and innovative engineering solutions.
328 postsWebsite
Latest articles
Shopify ·
React Native is the Future of Mobile at Shopify
Shopify is moving forward with React Native for new mobile apps after years of native development, driven by mobile purchasing volumes and a desire to consolidate iOS and Android work. It describes React Native as a framework that uses JavaScript and React’s declarative model to translate virtual DOM structures into native platform views through native bindings. Experiments with Arrive, Point of Sale, and Compass reported twice the productivity on Arrive, an estimated 80% code sharing target, and observed sharing of 95% for Arrive and 99% for Compass, while testing lowered the considered Android CPU threshold from 2GHz to 1.5GHz. The policy does not require rewriting existing native apps, leaves low-level libraries native and native hiring in place, and calls for dedicated React Native tooling, foundations, and open-source partnerships.
2023-10-18Shopify ·
Scaling Mobile Development by Treating Apps as Services
Shopify describes the challenge of scaling mobile development without slowing feature delivery as teams and projects adopt divergent tools, making knowledge sharing, cross-project contribution, and incident response harder. It extends ServicesDB, previously used to track production services, to treat mobile apps as services with owners and explicit expectations encoded in configuration files. Automatic checks cover ownership, contact information, CI and Shipit Mobile deployment configuration, internal-tool versioning, and production error and crash monitoring. Failed checks generate GitHub issues with remediation instructions, while the contract helps the Mobile Tooling team identify compatible projects and helps developers understand why tooling may not work. The implementation currently has basic checks, with planned work including mobile-app service-level objectives, easier project creation, and shorter build times.
2023-10-18Shopify ·
How to Implement a Secure Central Authentication Service in Six Steps
Shopify built Identity to reduce the overhead of managing separate staff accounts across multiple stores, where each Shop isolated user data and lacked single sign-on. The service uses OpenID Connect (OIDC) to centralize authentication while retaining destination-specific authorization, and it models legacy one-store accounts separately from Identity accounts that can access multiple destinations. Migration begins by synchronizing profile and security data, routing authentication through Identity, and prompting eligible users to verify email ownership, combine accounts, and provide required 2FA credentials. Single legacy accounts can be auto-upgraded without changing their UUID, while multi-account merges use a MergeSession and operations implemented with ActiveOperation; WebAuthn also supports physical security keys as a second factor. Finally, Shopify changed store signup and staff invitations to prevent new legacy accounts, reporting that more than 75% of active accounts had been upgraded or combined at the time of writing.
2023-10-18Shopify ·
How Shopify Manages API Versioning and Breaking Changes
Shopify describes how it versions APIs and manages breaking changes for an ecosystem of tens of thousands of partners. It releases date-based API versions every three months, with changes introduced in an unstable version and represented in the monolith through frozen records and a change file. Teams classify changes by whether third-party developers must migrate, use impact analysis and forward-compatibility testing, and protect uncertain changes with beta flags. For necessary breaking changes, tooling marks breaking or possibly breaking request paths, sends metadata through Monorail to a data warehouse, and produces reports on affected calls. Runtime conditionals then help teams manage old and new behavior, measure adoption, identify remaining risk, and communicate with developers.
2023-10-18Shopify ·
Successfully Merging the Work of 1000+ Developers
Shopify’s single monolith supports changes 40 times a day and roughly 400 commits to master daily, making safe trunk-based development difficult at the scale of 1,000+ developers. Its first Merge Queue limited deployment drift and batch size, but queued pull requests did not receive CI, soft conflicts could break master, and a browser extension enabled accidental direct merges. Merge Queue v2 replaces that workflow with GitHub comments and webhooks, programmatic branch protection, an emergency /shipit --emergency path, and a predictive branch where queued changes receive CI before merging. A reconciliation algorithm synchronizes that branch through GitHub’s GraphQL API, while failure-tolerance thresholds handle flaky tests and remove repeatedly failing pull requests. CI starts when requests enter the queue; batches of eight and three concurrently tested batches balance throughput, risk, and resource use, while the changes improved user experience, deployment safety, and deploy throughput.
2023-10-18Shopify ·
Four Steps to Creating Effective Game Day Tests
Game Day tests deliberately trigger failure modes in production systems and examine whether automated behavior and human response match expectations. Shopify frames the practice as preparation for outages that can arise despite code checks, reviews, and other safeguards. The method starts by inventorying possible failures, then selecting scenarios by impact and simulation difficulty rather than estimated likelihood. Each scenario becomes a controlled experiment with prerequisites, testable hypotheses, expected alerts and downstream behavior, plus recovery steps recorded in a spreadsheet. During execution, teams track results, communication, and work distribution; afterward they fix gaps, review coordination, update response guidelines, and rerun tests to verify changes, with periodic testing keeping systems and people prepared.
2023-10-18Shopify ·
Sam Saffron AMA: Performance and Monitoring with Ruby
Sam Saffron discusses Discourse’s approach to Ruby performance and monitoring, arguing that production memory constraints matter more than isolated microbenchmarks for many deployments. He favors clear code and selective optimization, sometimes bypassing ActiveRecord with MiniSql on performance-sensitive paths rather than broadly sacrificing Ruby’s readability. Discourse keeps performance under continuous observation, and Saffron describes budgets for dependencies, boot time, and high-profile pages, including alerts for query-count regressions. He identifies memory leaks as especially difficult to diagnose, describes bisecting an application to isolate a V8-Ruby interop leak, and says MRI remains the only feasible runtime for Discourse while memory profiling and analysis tooling remain substantially behind Java and .NET.
2023-10-18Shopify ·
Make Great Decisions Quickly with TOMASP
Technical leaders and managers face high-impact decisions about hiring, technology, architecture, and priorities, but cognitive biases, short-term emotion, and overconfidence can distort judgment. TOMASP is a framework for making decisions timely, considering alternatives, defining decision criteria, examining uncertainty, and preparing to be wrong. It recommends timeboxing the decision, generating at least three options, making a meta-decision about optimization criteria, analyzing options, stepping back from emotion, and preparing to be wrong through prototypes and tripwires. In the Flutter rewrite example, Michelle compares alternatives against cross-platform consistency, performance, and development speed, then sets a two-week prototype and blind test. The framework treats decision quality as a function of how a choice is made and implemented, not simply its eventual outcomes.
2023-10-18Shopify ·
Five Common Data Stores and When to Use Them
The article compares five common data stores—relational databases, NoSQL databases, key-value stores, full-text search engines, and message queues—and connects each to appropriate design needs. Relational databases use schema-bound tables, primary and foreign keys, SQL, and ACID transactions for durable business-critical data, while NoSQL databases trade strict schemas for flexibility with document and wide-column models. In-memory stores such as Redis and Memcached provide fast access and configurable eviction for caching; Redis additionally supports richer data structures and disk persistence. Full-text search engines target word and substring searches, whereas Kafka uses a distributed append-only log and producer-consumer model to move data reliably between services. The article concludes that the right choice depends on whether data must be permanent, flexible, quickly accessed, searched, or transferred.
2023-10-18Shopify ·
How to Write Fast Code in Ruby on Rails
Shopify’s guide presents performance advice for Ruby on Rails across Active Record, Rails, and Ruby, while treating speed as a feature rather than the first optimization priority. It recommends understanding Active Record’s lazy query execution, selecting fewer columns, avoiding unindexed queries, using safe indexing approaches for large tables, and treating query cache as short-lived rather than dependable. For Rails applications, it covers caching, throttling expensive or abusive operations, moving long-running work into Active Job-backed queues, and reducing dependency growth to limit boot time and memory use. Ruby-specific guidance includes limiting metaprogramming and indirection, choosing O(1) hash lookups over O(n) array searches when appropriate, and reducing allocations while avoiding harmful global mutation; benchmark figures show method-definition and invocation costs can differ.
2023-10-18Shopify ·
How Shopify Manages Petabyte Scale MySQL Backup and Restore
Shopify manages a petabyte-scale fleet of MySQL servers across replica-sets, or shards, in three Google Cloud Platform regions, where file-based Xtrabackup archives made shard restores take more than six hours. To reduce that Recovery Time Objective, the team built Kubernetes CronJobs around the Compute Engine Persistent Disk snapshot API, scheduling snapshots every 15 minutes while accounting for regions, zones, instance roles, and MySQL consistency variables. Persistent Disk snapshots take about 20 minutes initially and typically under 10 minutes incrementally; restoring the latest snapshot to a new volume and starting MySQL, including InnoDB instance recovery and replication-lag recovery, typically brings RTO below 30 minutes. Retention tooling keeps the latest two snapshots plus policy-defined dailies and weeklies, while verification checks startup, GTID auto-positioning, and InnoDB page corruption; offsite tooling compresses, encrypts, and transfers exported data to another provider.
2023-10-18Shopify ·
How Shopify Scales Up Its Development Teams
The article argues that scaling development is primarily about creating more effective teams, not simply increasing headcount. It first recommends checking whether an existing team prioritizes impactful work, minimizes supporting activity, and can ship code quickly while finding bugs; otherwise hiring may be premature. Shopify organizes product work into problem spaces and teams of five to nine, uses a multi-step process combining technical exercises with cultural assessment, and aims for at least as many onboarded members as onboarding members. New teams receive a baseline for work practices and technology, while transparent technical design, shared tooling with dashboards and embedded quality and testing standards, optional pairing, and frequent retrospectives support consistency and learning. The conclusion is that these practices, rather than hiring alone, enable development teams to scale more effectively.
2023-10-18Shopify ·
Want to Improve UI Performance? Start by Understanding Your User
Shopify’s team examined UI performance in the Marketing section of the Shopify admin, aiming to improve load time, perceived load time, and interactions after navigation. They tested the experience on a low-end Moto G3, profiled browser activity with Chrome Developer Tools and React DevTools, and used merchant workflows to reorder priorities. The work reduced unnecessary initial work by memoizing repeated date-formatting tasks, deferring the nonessential ImagePicker bundle, and prefetching that component and its data when a merchant hovered over its activator; memoization cut one component’s render time from about 64.7 ms to 0.5 ms. For loading states, the team recommends stable skeletons with static headings and immediately useful controls, such as the Create campaign button, so merchants can act before unrelated data arrives.
2023-10-18Shopify ·
Building Resilient GraphQL APIs Using Idempotency
Shopify’s Payment Service uses API-level idempotency to make GraphQL payment mutations resilient to timeouts, connection failures, and ambiguous outcomes that could otherwise leave merchants unpaid or double-charge buyers. The design places a UUID-based idempotency key directly in each mutation input, identifies requests by client plus key, and locks concurrent duplicates so retries receive a 409 response while the original call is processing. An IncomingRequest record stores request state and completed recovery points, allowing completed responses to be replayed and incomplete attempts to recover before continuing. Handlers divide work into no-side-effect, transactional local-side-effect, and remote-side-effect steps, each optionally providing run and recover functions. The approach adds database-write overhead and requires careful compatibility testing when recovery points change, but the team reports that its structured handlers are clear and maintainable.
2023-10-18Shopify ·
Living on the Edge of Rails
Shopify describes running its Core monolith on the unreleased Rails 6 master branch, or “Edge of Rails,” since February 2019, with the goal of continuously using the framework’s latest revision. Instead of annual upgrades, weekly changes reduce diff size and let developers use upstream features quickly, while a 130,000-plus-test suite finds edge cases and informs patches to Rails and related projects such as Zeitwerk. Solid Track automates weekly Rails bumps, opens GitHub pull requests, and triggers git bisect when CI fails to identify the breaking commit. Green builds move through ten-minute canary deployments before reaching the remaining servers. The approach trades the stability of released dependencies for faster fixes and features, relying on monitoring, tests, and controlled deployment to manage risk.
2023-10-18Shopify ·
Pagination with Relative Cursors
Incremental page-number pagination becomes increasingly expensive for large offsets, because the database scans and discards earlier records before returning a page. Shopify describes relative cursor pagination, which remembers the last record seen and continues from that point, using an indexed id to keep query time nearly constant as pages increase. In tests on a shop with 14 million products, a 100,000 offset took 2,221.60 ms, compared with 5.24 ms using the last id, while offsets of 1,000,000 and above consistently timed out. Sorting by another field requires retaining both its last value and a unique secondary key such as id, with a matching compound index to avoid skipped records and maintain performance. Shopify introduced Link-header URLs for next and previous pages and began migrating REST endpoints from page numbers, reporting early /admin/products.json usage as about 11 times faster, though adoption remained low and full results were pending.
2023-10-18Shopify ·
Lessons from Leading a Remote Engineering Team
Managing remote engineering teams can expand hiring beyond office locations, support a more diverse workforce, retain employees through relocations, and enable distributed coverage such as 24/7 support. The guidance describes building such teams around relationships, trust, and communication, with “Default to open internally” and “Charge your trust battery” as supporting cultural values. It recommends asynchronous documentation and discussion in email, GitHub issues, Google Docs, and project systems, supplemented by a fixed weekly synchronous meeting and shared core hours. Managers should also support healthy home workspaces, maintain regular one-on-ones and timely contact, arrange selective in-person sessions, and actively champion remote employees’ visibility, recognition, and career progression. The approach requires deliberate effort but can provide hiring, composition, retention, and strategic benefits.
2023-10-18Shopify ·
Componentizing Shopify’s Tax Engine
Shopify’s tax engine grew from a simple system into difficult code as tax rules and ownership spread across Cart, Checkout, and Order. The team first learned the surrounding code and tax domain with help from a tax subject matter expert, then mapped dependencies using regular expressions, scripts, manual investigation, and tax-related bug work. They moved tax source files into a dedicated component behind explicit TaxesRequestSchema and TaxesReponseSchema interfaces, with simple-type request and response schemas, protected boundaries, and expanded unit, integration, and end-to-end tests. During release, the old and new engines calculated taxes concurrently so discrepancies could be measured; after iterative fixes, Shopify rolled the component out gradually from a small group of shops to all shops over several days, with no downtime or merchant impact reported.
2023-10-18Shopify ·
Implementing Android POS Receipt Printing on Shopify
Shopify’s Android POS lacked printed-receipt support until March 2019, leaving merchants who needed it to use iPads. The design targets a pragmatic, adaptive, composable, maintainable, and highly testable API, while handling more than 8 billion possible receipt content combinations across discounts, tips, transactions, gift cards, refunds, payment methods, and regional requirements. Its six-step pipeline aggregates models from SQLite or GraphQL, transforms them into a PrintableReceipt, defines view-based sections, creates printer-agnostic PrintRequest objects, renders section bitmaps at the required resolution, and passes them through the Hardware SDK to vendor-specific integrations. The PrintableReceiptComposer contains most business logic, with unit tests covering features and edge cases, while screenshot tests compare rendered output pixel by pixel in CI. This componentized separation isolates printer support from receipt generation and allows new printer models to be added in a couple of hours.
2023-10-18Shopify ·
Mobile Release Engineering at Scale with Shipit Mobile
Shopify built Shipit Mobile to replace error-prone, manual mobile app releases with a shared platform for creating, viewing, and managing releases. The process creates a release branch from master, builds a candidate for each commit, increments build numbers, and starts separate CI pipelines for testing and release builds. Shipit Mobile uploads the same binary and metadata to distribution channels including Google Play, the App Store, and internal support access, using store APIs and Fastlane, while Slack notifications make release status visible beyond the release captain. Developers can bypass CI when necessary for emergencies, and configuration favors convention over configuration for project metadata. After more than six months in production, the platform reduced release complexity, shortened the release cadence from three weeks to one week, and helped more people become release captains.
2023-10-18