Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,328 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
Modelling Developer Infrastructure Teams
Developer Infrastructure at Shopify spans local and cloud development environments, CI systems, deploys, and technology-specific foundations for web and mobile applications. The post models these teams against a three-phase develop–validate–deploy workflow, distinguishing horizontally integrated teams that own one phase from vertically integrated teams whose systems span phases for a particular technology. It uses this model to identify dependencies, gaps, differing scopes, collaboration patterns, and three work avenues—concentration, expansion, and interfacing—with the mission of improving and scaling the cycle. The analysis informed organizational changes, including a Client Foundations team, closer collaboration, and plans for workflow-wide user research focused on the developer journey from writing a patch to production.
Bridging the Gap Between Developers and End Users
Shopify describes how distance between developers and end users can weaken product context and empathy as companies grow. The company created Bridge the Gap in 2014, connecting employees with merchants through Support teams. The program uses workshops, an internal podcast built from support-call clips and discussions, and voluntary one-hour shadowing sessions in which participants observe Support Advisors helping merchants via chat, phone, or email. Direct exposure is presented as a way to understand unexpected product use, identify customer workarounds that may inform fixes or new products, broaden domain knowledge, and develop listening, communication, patience, empathy, and expectation-setting skills. The broader goal is to make merchants accessible to Shopify’s workforce as people rather than abstract accounts or personas.
Connecting with Mob Programming
A six-person team drawn from three teams had six weeks to complete an ambitious Shipping project, while several members were new to the domain or company and had not worked together. Pairing was uncommon, and pull requests received little substantive feedback, creating uncertainty about whether the team was growing, connecting, and learning. After Sheldon Nunes introduced mob programming, the team used a shared screen and ten-minute driver rotations, with each driver pushing changes before the next person pulled them. This kept everyone engaged, accelerated reversible decisions, limited context dumps, and made most code review back-and-forth unnecessary. Across nearly daily sessions for three months, the team reported stronger relationships, easier feedback, restored remote learning, and better support for newcomers, while identifying drawing, public discoverable calls, simple environments, cross-disciplinary participation, and breaks as important practices; tests and documentation were considered poor fits for mobbing.
Spot: a video sharing app built with Flutter
Supabase compiles a broad index of case studies documenting how organizations deploy its platform for infrastructure, authentication, and database workloads across diverse industries. Multiple companies report significant operational efficiencies, including Quilia reducing development time by seventy-five percent through Data API and row-level security capabilities. Infrastructure migrations are highlighted prominently, with Shotgun achieving an eighty-three percent reduction in data infrastructure costs and Good Tape cutting expenses by sixty percent. Several teams also adopt the platform specifically for artificial intelligence workflows, such as Quivr deploying thousands of vector databases and Firecrawl transitioning from Pinecone to Supabase Vector. Other organizations, including Mobbin and Replenysh, use the platform to migrate hundreds of thousands of user accounts and implement one-time password systems within twenty-four hours.
A Guide to Running an Engineering Program
Shopify describes a playbook for running large engineering programs as platform complexity grows across products, tooling, and architecture. The framework begins with a Program Plan covering the problem statement, objectives, guiding principles, definition of done, risks, mitigation, staffing, scope, and a path to completion. Execution uses six-week cycles that revisit unfinished goals, regressions, emerging risks, and the planned route to completion; the described program spans 200 people, nine sub-organizations, and 92 projects. Supporting rituals include weekly updates, status checks, risk and escalation triage, cycle reviews, retrospectives, RFCs, and performance testing with Lua scripts orchestrated by Genghis. The post concludes that engineering program management must adapt to each organization and that a combination of scheduled and ad hoc rituals helped Shopify pursue its goals.
Processing ETL tasks with Ratchet
Grab's Lending team relies on multiple microservices to finance various user segments, requiring automated pipelines for operational tasks like loan rescheduling and merchant whitelisting. To execute these workflows, the team uses Ratchet, a Go library that organizes data pipelines into concurrent stages connected by Go channels and JSON payloads. In the Azkaban service, each ETL task is structured as a Job Processor that configures custom Data Processors for extraction, transformation, and loading. For merchant whitelisting, a custom processor pulls uploaded CSV files from Amazon S3, validates the rows to prevent manual errors, calls microservice APIs, and passes the output to an empty load stage. This pipeline architecture allows the team to isolate failures quickly, eliminate manual data validation, and automate routine operational tasks.
Perspectives on React Native from Three Shopify Developers
Three Shopify developers—web developer AJ Robidas, Android developer Michelle Fernandez, and iOS developer Ash Furrow—describe how React Native changes mobile development for people from different backgrounds. They say its React-based model lets web developers build for iOS and Android, helps native developers create interfaces faster and reach a wider audience, and supports collaboration across web, Android, and iOS teams. The developers discuss surprises including React Native’s small set of building blocks, fast UI feedback, native bridge modules, existing SDK integration, and apps that can be difficult to distinguish from Swift or Objective-C software. They also identify challenges such as mobile-specific performance concerns across the RN Bridge, the Metro and Node ecosystem, and the large dependency stack. At Shopify, Visual Studio Code, command-line tooling, internal foundations, workshops, and cross-disciplinary teams support adoption.
Shopify-Made Patterns in Our Rails Apps
Shopify describes patterns and infrastructure adaptations used to keep its large Rails monolith maintainable as its organization and platform scale. Its dev command-line tool standardizes project setup, dependencies, services, environment variables, integrations, and command aliases across hundreds of active projects. Shopify Core uses pods to partition shops across independent databases and extends the shard concept across provisioning, deployment, load balancers, caching, and servers, while splitting code into domains without separate databases. The post also details safeguards including short-term Active Record migrations consolidated periodically into structure.sql, automated tests, a pod-aware maintenance task system, and a translation platform that commits translated strings through Git. Finally, Monorail imposes versioned JSON event schemas between Kafka producers and consumers and supports review and automated scrubbing of PII fields.
High Availability by Offloading Work Into the Background
Unpredictable traffic spikes, slow third-party payment requests, and resource-intensive image processing can overwhelm an application, making it slow or unavailable; the text presents background jobs as a way to preserve meaningful user interactions. High availability is defined not merely as server uptime, but as users being able to interact with the application when needed. The application server places a task message in a queue and continues serving requests, while workers process jobs asynchronously, with support for retries, parallelization, prioritization, and event- or time-based scheduling. Offloading improves response times and buffers spikes, but changes immediate guarantees: an upload may be accepted before processing succeeds, and errors may be handled later. For distributed systems, data-based messaging such as RabbitMQ can connect different services, while Kafka's append-only, replayable event log supports shared consumption and event sourcing.
Understanding GraphQL for Beginners–Part One
The article introduces GraphQL as an open-source data query and manipulation language for APIs, contrasting it with REST's fixed response shapes and multiple endpoint calls. GraphQL lets clients describe exactly which fields they need, returning that requested structure through a single endpoint rather than a cookie-cutter response. Using a Toronto Eagles example, the article explains how one GraphQL query can combine requests for a team's founding year, captain's name, and last championship, avoiding three separate RESTful trips whose latency can vary. It also uses burger and bank withdrawal analogies to explain over-fetching, under-fetching, flexible frontend iteration, and fewer endpoints, then points readers to a forthcoming Ruby on Rails implementation in part two.
Let’s Encrypt x Shopify: Securing the Web 4.5 Million Domains at a Time
Shopify describes how its SSL team moved merchants’ stores to HTTPS and ultimately secured more than 4.5 million domains with Let’s Encrypt. The effort began against certificate providers whose APIs were not designed for fully automated provisioning: errors often arrived as human-readable messages, and imposed throttling could have stretched initial issuance to about 100 days. Shopify tested the ACME protocol, valued its standardized specification and open-source implementation, and first added Let’s Encrypt as a backup certificate authority to improve emergency key or certificate-chain rotation. The team rolled out that integration in a few hours, compared with months for earlier providers, then made Let’s Encrypt its primary authority after finding it reliable. The account presents this change as evidence that ACME can make certificate automation more responsive and allow certificate authorities to be added without redesigning Shopify’s infrastructure.
Rate Limiting GraphQL APIs by Calculating Query Complexity
Shopify’s GraphQL Admin API uses calculated query costs to address two limitations of request-based rate limiting: identical credits for responses of different sizes and equal treatment of reads and side-effecting writes. The server statically analyzes each query, assigning one point to objects and interfaces or unions, zero to scalars and enums, two plus the requested object count to connections, and 10 to mutations. Clients receive 50 points per second up to a 1,000-point limit, while responses expose requested and actual costs; when fewer connection records are returned, the difference is refunded. The post reports a linear correlation between calculated query complexity and execution time, giving Shopify more predictable load for infrastructure scaling and a way to identify performance outliers.
10 Lessons Learned From Online Experiments
After completing ten online experiments over the past year, the author describes recurring statistical and implementation traps that can distort experiment results. The discussion stresses choosing independent randomization units, examining whether the analysis unit matches the randomization unit, and using simulations to test suspected false-positive effects before changing an experiment plan. It also shows how request-level traffic analysis and close collaboration with engineering can avoid unnecessary implementation work, while generous logging, A/A tests, and investigation of sample ratio mismatches can expose redirect failures or qualification bugs. The author reports that switching one experiment’s analysis unit from session to user changed its result from negative to positive, and argues for monitoring results without stopping early, investigating anomalies, and slicing averages across important user segments.
How Shopify Built An In-Context Analytics Experience
Shopify’s Analytics section gave merchants access to business data, but research and quantitative analysis found that they often moved between it and admin pages while making decisions. The team therefore designed an in-context experience for the Orders page, selecting metrics through workflow analysis, segmenting merchants by factors such as order volume and fulfillment process, and examining each metric’s usefulness, variability, and time horizon. Data scientists prototyped visualizations with real, segmented data and created a specification sheet so Web, Android, and iOS used consistent queries, data sources, fields, and aggregations; the project also tested batch models before streaming them. The launched experience surfaces order and fulfillment metrics for daily, seven-day, and thirty-day periods, with comparison indicators, and links to reports covering orders, returns, products, and fulfillment, shipping, and delivery times.
Elixir at Ramp 🧪
Ramp engineer Pablo describes why Elixir suited several critical services, while arguing that technology choice is only one part of engineering success. He contrasts BEAM, Erlang’s virtual machine, and OTP’s abstractions for reliable, concurrent systems with the adoption and developer-experience advantages of Elixir, including familiar syntax, Mix, iex, hot code reloading, doctests, macros, and sigils. Immutability and aliased module calls are presented as safeguards against hidden state changes and circular-import problems in growing codebases, though the post also notes that Erlang and Elixir lag behind bolted-on type checkers. Ramp uses Elixir for its real-time Authorization engine and Risk analysis platform; the Authorization service is described as exceptionally stable and flexible for rapid feature development. The company’s largest backend remains Python, and the conclusion prioritizes sleep, trust, ownership, and psychological safety over any specific stack.
Other Driven Developments
The article argues that development models turn implicit lessons into shared, conscious practices as teams, codebases, and responsibilities grow. It presents Grep Driven Development, emphasizing expressive and consistent names so behavior remains searchable, refactorable, abstractable, and predictable, while warning that premature abstraction and locally “improved” naming can hinder later change. It also makes the case for Copy/Paste Driven Development as a deliberate, judgment-based tool, noting that duplication can be cheaper than the wrong abstraction and that abstractions with flags and conditionals may create their own costs. The article names Ego Driven Development and Stickler Driven Development among its four approaches, then concludes with Human Driven Development: choosing, bending, or breaking rules to make work easier for present and future teammates.
Fly's Prometheus Metrics
Fly describes the Prometheus-style metrics infrastructure behind its globally distributed micro-VM platform and explains why it favors metrics over traditional monitoring checks. The system collects counters and other measurements through HTTP /metrics endpoints, stores them in a clustered VictoriaMetrics (“Vicky”) deployment, and uses Telegraf, Vector, exporters, and Consul to connect sources and manage discovery. Within Firecracker instances, custom init, the Nomad driver, and a vsock carry node-exporter-type JSON statistics from guests for collection, while applications can expose their own exporters. Deployments can declare an exporter port and /metrics path in fly.toml, after which Fly routes those application metrics into Vicky alongside platform data. The resulting Prometheus API is available for Grafana queries, dashboards, and alerts, with labels such as app, region, host, and instance; the post argues that the same historical data supports anomaly detection, trending, and planning.
Three Ways We Share Context at Shopify Engineering
Shopify Engineering treats context—the why, what, and how behind development philosophies, approaches, and choices—as essential to company, team, and individual growth. It describes the Development Handbook, which evolved from a 2015 Google Doc into a searchable internal website in 2018, with hundreds of developer-contributed topics maintained through GitHub pull requests, reviews, and Markdown. Dev Talks began as a weekly open stage and became an on-demand recording program after the 2020 shift to Digital by Design, with most presentations lasting 15 to 20 minutes. The Technical Leadership Team’s Shift podcast provides monthly, in-depth discussions of technical decisions, distributed in audio and video with transcripts. The programs continue to evolve while facing content-maintenance and participation challenges.
How I Define My Boundaries to Prevent Burnout
The post describes how an intense, single-tasking software leader developed a sustainable way to divide work, family, and recovery time. After 100-hour startup weeks caused scattered thinking, dulled analytical ability, wrist problems, and an inability to type, the author found that a focused 60-hour, 9-to-9 schedule delivered 80 to 90% productivity compared with 100 hours, then shifted to a 9-to-6 model in 2009. The approach separates concentrated work from non-work, schedules blocks for writing, planning, reviewing, and thinking, and uses email and Slack response patterns to communicate boundaries and expectations. It lets lower-priority tasks fall away instead of extending work indefinitely into personal hours. The author says this structure supports recovery, family presence, and prioritization, while emphasizing that each person should experiment with a sustainable model rather than treat nine-to-six as universal.
How Shopify Dynamically Routes Storefront Traffic
Shopify’s Storefront Renderer team rewrote the storefront implementation beginning in 2019 and needed a way to shift traffic between the legacy and new versions while migrating features. Using nginx with OpenResty, Lua modules, a control plane, and the spy chatbot, they moved routing rules from a Lua file into JSON payloads that could be created, updated, enabled, disabled, and deleted without a CI or deployment cycle. Rules selected shops and request subsets through filters, then controlled rendering and verification rates, including forward verification, reverse verification, and self-verification in the nearest region. During nginx request processing, matching occurs in the rewrite phase; verification is scheduled with timers in the log phase so duplicate requests run asynchronously after the client response path. The system supported gradual rollouts, parity checks, and resilience testing under Redis failures, while the team planned to simplify routing after the new storefront served almost all traffic.
Top topics
- Architecture830
- Performance812
- LLMs655
- Machine Learning619
- Open Source562
- AI Agents546
- AI458
- Developer Experience447
Weekly digest
The best engineering reads, once a week.