The Journey to Cloud Development: How Shopify Went All-in on Spin

Summary

Shopify’s development environment, the Ruby-based dev tool and xhyve-backed railgun VM, no longer handled the growing number of repositories, services, integrations, and resource demands around its monoliths. Developers needed more services, faced sluggish code-build-test loops, and encountered costly state changes when tophatting collaborators’ work. The Environments team concluded that development environments needed cloud-like elasticity, then tested an automatically configured local Kubernetes cluster and easy creation of Google Compute Engine VMs; the source also describes Isospin and Spin as the resulting direction. Local Kubernetes did not gain broad organic traction, while the current Isospin implementation is described as the team’s best iteration for developers working on majestic monoliths. The team plans to evolve it into an application-development library built on dev, add dependency inference, and address the tension between persistent environments and Kubernetes pod relocation.

Context

Shopify’s dev tool and laptop-based environments had been outgrown by increasingly complex projects involving multiple repositories, running services, integrations, and monoliths. Developers needed more supporting services, but limited laptop resources made environments sluggish and extended code-build-test and shipping cycles. Tophatting also required repeatedly changing application and database state, while teams developed ad hoc ways to handle combinations of feature branches and services.

Approach / What changed

The Environments team investigated more elastic, cloud-oriented development environments through tightly scoped experiments, including an automatically configured local Kubernetes cluster and easy creation of Google Compute Engine VMs. The later Isospin implementation provides a development environment that feels like a small Linux box. The team plans to build an application-development library on dev, infer dependencies from repositories, and investigate the persistence and ephemerality trade-off of running instances as Kubernetes pods.

Takeaways

• Shopify’s laptop-based dev environment struggled with the resource demands of additional services and integrations around large Rails monoliths, slowing developers’ code-build-test loops.
• An automatically configured local Kubernetes experiment did not gain organic traction among most developers, despite interest from Kubernetes-focused teams and early adopters.
• The planned evolution of Isospin includes repository-based dependency inference and work on the tension between persistent environments and Kubernetes pod relocation.

Related reading