---
title: "Migrations"
description: "134 posts about Migrations, summarised, each linking to the original."
---

# Migrations
> 134 posts about Migrations, summarised, each linking to the original.

## Articles

### [Caching Without Marshal Part 2: The Path to MessagePack](https://yomu.fyi/post/caching-without-marshal-part-2-the-path-to-messagepack.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 15, 2022
- Type: Problem & solution

Rails caching relied heavily on Marshal, but cached Ruby-specific objects could break after code changes when old payloads were loaded by code without the former classes. MessagePack provided a generic binary format whose core types exclude Ruby-specific Object and instance-variable encodings, while extension types allowed production-specific serializers such as the compact Date representation. During a roughly six-month migration, the team ran both formats, prefixed MessagePack payloads with a version byte, and used encoding failures to identify unsupported cache values. To scale beyond 128 extension types, they added an Object catchall based on as\_pack and from\_pack, plus Struct and T::Struct modules whose attribute digests detect refactors and turn stale data into cache misses. After logs showed that Marshal was no longer used, the core monolith moved exclusively to MessagePack, with the Paquito gem extracting much of the migration work.


### [Supabase Beta September 2022](https://yomu.fyi/post/supabase-beta-september-2022.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: ant\_wilson
- Published: Oct 5, 2022
- Type: Announcement

Supabase’s September 2022 update centers on reducing technical debt and clearing its backlog after five Launch Weeks. The team ran three Kaizen weeks focused respectively on QA and testing, documentation, and issue backlog and automation, closing over 250 open issues and 50 pull requests while pausing feature development. Product releases included Auth UI, a pre-built React component for Supabase Auth with custom themes and extensible styles, plus an open-source Postgres WASM release with Snaplet for running Postgres in a browser through an embeddable Linux Virtual Machine. Security changes to Dashboard database interactions simplify permissions and reduce the security surface area; they were scheduled for automatic application in November or could be run earlier through the Dashboard. Other updates covered Edge Functions, documentation, migration tooling, and community projects.


### [Safely Adding NOT NULL Columns to Your Database Tables](https://yomu.fyi/post/safely-adding-not-null-columns-to-your-database-tables.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 4, 2022
- Type: Explainer

Shopify’s Database Migrations team investigated whether Large Hadron Migrator (LHM) can safely add NOT NULL columns to existing MySQL tables with minimal downtime. LHM creates a shadow table, installs triggers for INSERT, UPDATE, and DELETE operations, and copies records in batches before renaming the tables. The experiments varied whether the new column had a DEFAULT value or UNIQUE index and whether MySQL used strict or non-strict mode, evaluating trigger compatibility and record counts. A NOT NULL column with a DEFAULT was backward compatible, but adding a UNIQUE index could cause data loss; without a DEFAULT, strict mode broke compatibility while non-strict mode used implicit defaults, which may be undesirable.


### [Lessons From Building Android Widgets](https://yomu.fyi/post/lessons-from-building-android-widgets.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 22, 2022
- Type: Explainer

Shopify revisited its Android widgets alongside an iOS widget effort, aiming to give merchants faster access to frequently changing business metrics. The team chose native Android development because React Native lacked official RemoteViews support, while widget configuration data was persisted with shared preferences and used to build requests. Because Android could invoke onUpdate() multiple times per cycle, they added a one-minute, short-lived cache keyed by widgetID and mindful of configuration such as locale. This reduced redundant network calls, system load, and incorrect analytics, while the widget rollout also exposed migration limitations between old fixed-size widgets and new space-responsive ones, leading Shopify to deprecate the old widgets rather than transform them.


### [Choosing a Postgres Primary Key](https://yomu.fyi/post/choosing-a-postgres-primary-key.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: victor
- Published: Sep 8, 2022
- Type: Explainer

Choosing a Postgres primary key involves balancing simplicity, storage, sortability, standards compliance, information leakage, and collision avoidance. The discussion starts with natural and surrogate keys, then compares integer and biginteger identity columns with serial sequences, noting that sequences can have gaps and exposed numeric IDs may reveal information. It then examines pseudorandom and secure random numeric IDs before turning to UUIDs, including UUIDv4 and time-based alternatives, whose generation methods and index behavior differ. Postgres integration, size queries, and the pg\_idkit library provide practical ways to generate and evaluate identifiers; the suggested rules of thumb favor integers for simplicity, UUIDv4 for collision avoidance, xid for time-based sort friendliness, and UUIDv6 or UUIDv7 for a more standards-oriented choice.


### [supabase-js v2](https://yomu.fyi/post/supabase-js-v2.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: inian,alaister
- Published: Aug 16, 2022
- Type: Announcement

Supabase announces supabase-js v2, a release focused on developer quality-of-life improvements and designed around incremental changes rather than a large rewrite. Its type system uses CLI-generated database definitions at the client level, while authentication adds explicit signInWithPassword() and signInWithOtp() methods, makes most Auth operations asynchronous, and moves server-side methods under auth.admin. Data mutations return minimal by default unless select() is appended, constructor options are scoped by library, and Edge Functions automatically detect request and response content types. The release also introduces typed errors, a channel() interface for presence and PostgreSQL change subscriptions, and a preparatory path toward multiplayer features; older realtime syntax remains supported for now. Developers can install v2, consult updated documentation and migration guidance, while v1 continues receiving security fixes and maintenance patches for three months.


### [Supabase CLI v1 and Management API Beta](https://yomu.fyi/post/supabase-cli-v1-and-management-api-beta.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: soedirgo,qiao
- Published: Aug 15, 2022
- Type: Announcement

Supabase CLI v1 is moving out of beta with capabilities for managing database migrations and generating TypeScript types, alongside a beta Management API. The CLI can manage organizations and projects, link projects locally, run schema and manual migration workflows, support CI/CD through local and GitHub Actions workflows, and generate types from local or remote databases. Migration changes include improved manually written migrations, an experimental migra-backed db diff that is faster and produces more concise DDL, and local branching for testing DDL without data loss; the source notes that generated scripts are not perfect. The REST-based Management API, previously called the Admin API, is consumed by the CLI and exposes programmatic management of organizations, projects, Edge Functions, and more, with endpoints intended for CI/CD and test-environment creation.


### [How Mike Lyndon is using Supabase to accelerate development of AllPullTogether](https://yomu.fyi/post/how-mike-lyndon-is-using-supabase-to-accelerate-development-of-allpull.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: May 26, 2022
- Type: Explainer

A collection of case studies outlines how diverse engineering teams adopt Supabase across various infrastructure, authentication, and database workloads. Companies like Good Tape and Shotgun achieved substantial cost reductions between 60% and 83% by migrating to Supabase managed Postgres and authentication services. For security and data access control, Quilia reduced development time by 75% leveraging Supabase Data API alongside Row Level Security features. In machine learning applications, organizations such as Firecrawl, Berri AI, and Quivr utilize Supabase Vector for PostgreSQL vector embeddings rather than standalone vector databases. Additional migrations include Mobbin transferring 200,000 users from Firebase for authentication, while teams like Replenysh implemented one-time password workflows in under 24 hours.


### [Building a Business System Integration and Automation Platform at Shopify](https://yomu.fyi/post/building-a-business-system-integration-and-automation-platform-at-shop.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 22, 2022
- Type: Problem & solution

Shopify needed a maintainable way to connect increasingly modular business systems whose interfaces ranged from SOAP and FTP to REST and GraphQL. The company evaluated Mulesoft but found that its proprietary XML and Java virtual machine stack isolated the team from Shopify’s development ecosystem and complicated access to internal services and infrastructure. It instead built a Ruby on Rails integration platform around minimal validation, transparent data transformations, flexible client support, security, reusable components, documentation, and request-level testing with WebMock. The implementation favors direct manipulation of JSON hashes because that keeps mappings visible, while acknowledging risks such as typos and complex mock management. After a year, the platform had 18 integrations, had migrated all Mulesoft applications, and enabled other teams to build integrations, with ongoing work on testing and onboarding.


### [Fixing the Dependency Confusion Vulnerability in 600+ Ruby Apps - Shopify](https://yomu.fyi/post/fixing-the-dependency-confusion-vulnerability-in-600-ruby-apps-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jan 27, 2022
- Type: Problem & solution

Shopify identified a dependency confusion vulnerability through its bug bounty program; an attacker could exploit ambiguous package sources to install a malicious dependency and gain access to local, CI/CD, and production environments. To assess exposure, the Ruby Conventions team collected Bundler versions from repositories in a data lake and found around 600 susceptible Ruby applications. An experimental Bundler plugin converted Gemfile.lock files to the explicit-source format while restoring existing gem versions, and a CI-backed CLI later provided per-repository environments, parallel execution, pull requests, and easier failure tracking. After incremental migrations exposed configuration and tooling issues, Shopify updated supporting systems, set Bundler requirements, changed local and CI defaults, and worked with Bundler maintainers; the company reports migrating all its applications in less than a day and contributing to Bundler 2.2.21 and 2.2.22.


### [Upgrading MySQL at Shopify](https://yomu.fyi/post/upgrading-mysql-at-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 2, 2021
- Type: Problem & solution

In September 2021, Shopify completed its cutover from Percona Server 5.7.21 to 5.7.32 across a MySQL fleet spanning hundreds of horizontally sharded databases and thousands of virtual machines. The upgrade targeted replication improvements from transaction writesets, along with security and stability fixes, while avoiding the larger risks of a major-version change. The Database Platform team extended Mason into a declarative topology system and used Chef roles to provision upgraded VMs, restore backups, run mysql\_upgrade when required, configure replication, and populate replacement topologies. Extensive rollback testing exposed a critical downgrade bug; after patching 5.7.21, Shopify proceeded in staggered batches beginning August 25 and completed the final shards on September 7, while documenting a repeatable process intended to make future upgrades faster and safer.


### [Replenysh uses Supabase to implement OTP in less than 24-hours](https://yomu.fyi/post/replenysh-uses-supabase-to-implement-otp-in-less-than-24-hours.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Oct 19, 2021
- Type: Problem & solution

Replenysh builds infrastructure connecting brands with local communities to track, monetize, and recover used packaging and materials. The engineering team previously maintained their own backend infrastructure, pairing custom Twilio and password authentication with a Haskell server and a Postgres database on Heroku. Seeking to accelerate development velocity and minimize ongoing DevOps maintenance during a codebase refactor, Replenysh required an alternative backend platform that preserved relational Postgres and Row Level Security. The team selected Supabase over alternatives like Firebase because it provided native Row Level Security alongside built-in authentication tooling. By adopting Supabase Auth, Replenysh successfully implemented an SMS one-time password system in less than 24 hours before deploying their mobile application to production.


### [Shard Balancing: Moving Shops Confidently with Zero-Downtime at Terabyte-scale](https://yomu.fyi/post/shard-balancing-moving-shops-confidently-with-zero-downtime-at-terabyt.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 24, 2021
- Type: Explainer

Shopify’s MySQL shards can become unevenly utilized as merchants grow after being assigned to arbitrary shards, increasing over-utilization risk while leaving other capacity underused. The post describes a strategy for selecting shop placements using historical utilization and traffic data, simulations, and potentially other metrics such as shop size, GMV, and move time. It then explains online shop migration with Ghostferry, which batch-copies records and tails MySQL’s binary log before a brief cutover updates the routing table and directs new writes to the target pod. Verification checks data completeness, integrity, and stale traffic before old data is pruned; the migration system’s central algorithm is also represented in a TLA+ formal specification. The stated outcome is a more balanced infrastructure that can isolate high-traffic merchants across pods with virtually zero consumer-facing downtime.


### [Mobbin uses Supabase to authenticate 200,000 users](https://yomu.fyi/post/mobbin-uses-supabase-to-authenticate-200-000-users.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Jul 28, 2021
- Type: Problem & solution

Mobbin originally launched its mobile design search platform using Firebase, quickly growing to hundreds of thousands of registered users. As the platform scaled, the team faced user friction because Firebase Authentication did not merge accounts sharing the same email address across different providers. Furthermore, mutative Firebase Function failures threatened document data integrity, and multi-round-trip document queries drove high monthly costs across five million API requests. To resolve these challenges, Mobbin migrated its backend to Supabase, adopting its GoTrue authentication service and underlying Postgres relational database. This migration resolved authentication conflicts, safeguarded data consistency, and reduced monthly infrastructure expenses since Supabase does not charge on a per-API-request basis.


### [Spot: a video sharing app built with Flutter](https://yomu.fyi/post/spot-a-video-sharing-app-built-with-flutter.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Jul 27, 2021
- Type: Explainer

Supabase compiles a broad index of case studies documenting how organizations deploy its platform for infrastructure, authentication, and database workloads across diverse industries. Multiple companies report significant operational efficiencies, including Quilia reducing development time by seventy-five percent through Data API and row-level security capabilities. Infrastructure migrations are highlighted prominently, with Shotgun achieving an eighty-three percent reduction in data infrastructure costs and Good Tape cutting expenses by sixty percent. Several teams also adopt the platform specifically for artificial intelligence workflows, such as Quivr deploying thousands of vector databases and Firecrawl transitioning from Pinecone to Supabase Vector. Other organizations, including Mobbin and Replenysh, use the platform to migrate hundreds of thousands of user accounts and implement one-time password systems within twenty-four hours.


### [How Shopify Dynamically Routes Storefront Traffic](https://yomu.fyi/post/how-shopify-dynamically-routes-storefront-traffic.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 9, 2021
- Type: Explainer

Shopify’s Storefront Renderer team rewrote the storefront implementation beginning in 2019 and needed a way to shift traffic between the legacy and new versions while migrating features. Using nginx with OpenResty, Lua modules, a control plane, and the spy chatbot, they moved routing rules from a Lua file into JSON payloads that could be created, updated, enabled, disabled, and deleted without a CI or deployment cycle. Rules selected shops and request subsets through filters, then controlled rendering and verification rates, including forward verification, reverse verification, and self-verification in the nearest region. During nginx request processing, matching occurs in the rewrite phase; verification is scheduled with timers in the log phase so duplicate requests run asynchronously after the client response path. The system supported gradual rollouts, parity checks, and resilience testing under Redis failures, while the team planned to simplify routing after the new storefront served almost all traffic.


### [Supabase Dot Com](https://yomu.fyi/post/supabase-dot-com.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Apr 2, 2021
- Type: Announcement

Supabase announced a migration of its primary website from supabase.io to supabase.com over the upcoming week. User API endpoints will remain unchanged on supabase.co, requiring no configuration adjustments for active applications. The company originally selected supabase.io as a meme-inspired placeholder after releasing a Postgres Realtime engine on GitHub in 2019 and failing to find an available dot com name. Following advice from Y Combinator to secure a dot com address, the founders bypassed domain brokers charging percentage fees and negotiated directly with the domain owner, whose business address was located one hundred meters from the founder's home. The migration updates public website destinations while preserving existing developer infrastructure unaffected.


### [Supabase CLI](https://yomu.fyi/post/supabase-cli.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Bobbie Soedirgo
- Published: Mar 31, 2021
- Type: Announcement

Supabase announces its CLI as the first step toward running Supabase locally, managing database migrations, generating types and API or validation schemas from a database, managing projects, and eventually pushing local changes to production. The local stack runs through Docker Compose, using tools described as 100% compatible with every Supabase project, and a React setup can initialize it with supabase init before supabase start. For schema changes, the post favors database diffing over accumulating manual SQL migrations and adds a CLI mode and Docker image for pgAdmin Schema Diff, which tracks view-owner changes needed for Row Level Security and handles domains. The supabase eject command outputs Docker Compose, Kong, and Postgres configuration for self-hosting. An update dated 15 August 2022 states that CLI V1 is generally available and that a Management API is in beta.


### [Toad, a link shortener with simple APIs for low-coders](https://yomu.fyi/post/toad-a-link-shortener-with-simple-apis-for-low-coders.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Mar 8, 2021
- Type: Explainer

Supabase compiles an overview of customer case studies documenting how organizations deploy its platform across production environments. Multiple companies report infrastructure migrations, including Firecrawl switching from Pinecone to Supabase Vector for PostgreSQL vector embeddings and Mobbin moving 200,000 users from Firebase. Engineering teams document operational gains and cost efficiencies, such as Shotgun achieving an 83% reduction in data infrastructure expenses and Quilia saving 75% in development time using Data API and RLS features. Other organizations highlight infrastructure performance improvements, including Good Tape recording a 60% cost reduction alongside database efficiency gains after migrating to Supabase managed Postgres and Authentication. Teams also report accelerated implementation speeds, with Replenysh delivering OTP functionality in less than 24 hours and Xendit releasing a full solution to production within one week.


### [Monitoro Built a Web Crawler Handling Millions of API Requests](https://yomu.fyi/post/monitoro-built-a-web-crawler-handling-millions-of-api-requests.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Dec 2, 2020
- Type: Announcement

Supabase showcases numerous customer case studies illustrating how companies across various industries utilize its platform to build products, migrate infrastructure, and improve operational metrics. Several organizations report substantial cost and efficiency improvements, such as Shotgun achieving an 83% data infrastructure cost reduction and Good Tape saving 60% after moving to managed Postgres and authentication. In terms of developer velocity, Kayhan Space experienced an 8x boost in speed, while Quilia reduced development time by 75% through Data API and Row Level Security features. Other highlighted migrations include Mobbin moving 200,000 users from Firebase, Bree transitioning from Fauna, and Firecrawl switching from Pinecone to Supabase Vector for PostgreSQL vector embeddings.


[Newer posts](https://yomu.fyi/topic/migration/page/5.md) · [Older posts](https://yomu.fyi/topic/migration/page/7.md)