---
title: "Migrations"
description: "134 posts about Migrations, summarised, each linking to the original."
---

# Migrations
> 134 posts about Migrations, summarised, each linking to the original.

## Articles

### [Mintlify boosts NPS 30% and saves 60% with real-time analytics on ClickHouse Cloud](https://yomu.fyi/post/mintlify-boosts-nps-30-and-saves-60-with-real-time-analytics-on-clickh.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: Apr 14, 2026
- Type: Problem & solution

Documentation platform Mintlify experienced major dashboard performance degradation and rate limiting under PostHog as multi-tenant analytics traffic grew rapidly. Because PostHog lacked incremental materialized views, queries scanned full raw datasets and took tens of seconds to load. Mintlify migrated its customer-facing analytics infrastructure to ClickHouse Cloud, ingesting events through Amazon MSK and ClickPipes into a single events table backing incremental materialized views. The team executed the transition over two weeks by dual-writing events and replicating the existing PostHog schema to preserve API surface compatibility. The migration reduced dashboard query latency to sub-one-second, decreased related support bug reports to zero, lifted NPS by an estimated 30%, and lowered analytics infrastructure costs by approximately 60%.


### [Zero downtime Upgrade: Yelp’s Cassandra 4.x Upgrade Story](https://yomu.fyi/post/zero-downtime-upgrade-yelp-s-cassandra-4-x-upgrade-story.md)
- Company: [Yelp](https://yomu.fyi/company/yelp.md)
- Author: Mark Surnin and Muhammad Junaid Muzammil, Software Engineer
- Published: Apr 7, 2026
- Type: Problem & solution

Yelp's Database Reliability Engineering team upgraded over a thousand Cassandra nodes from version 3.11 to 4.1 without downtime. The upgrade delivered reliability, observability, and performance improvements, including Java 11 support, configurable guardrails, and fixed incremental repairs. To avoid the high costs, consistency compromises, and data streaming delays of standing up separate datacenters, the team selected an in-place rolling upgrade strategy. The rollout orchestrated staged node updates alongside compatible versions of the Stargate proxy and custom data pipeline connectors without requiring client code modifications. Although Stargate 2.x introduced regressions on range and multi-partition queries that required downgrading to 1.x, the finished migration achieved up to a 58% reduction in p99 latency across key clusters.


### [How Respan is scaling LLM observability with ClickHouse Cloud](https://yomu.fyi/post/how-respan-is-scaling-llm-observability-with-clickhouse-cloud.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: Apr 6, 2026
- Type: Problem & solution

Respan scaled its LLM gateway and observability platform from hundreds of daily requests to 50 million daily events, causing its original Django and Postgres architecture to stall under transaction contention at 50 to 100 requests per second. To overcome Postgres write-ahead logging bottlenecks, the team migrated the ingestion and analytics pipelines to ClickHouse Cloud. Engineers minimized row sizes by truncating large prompt text and writing structured metrics to MergeTree tables, then built incremental minute- and hour-level materialized views alongside scheduled refreshable views for user metadata. Distributed tracing spans are ingested asynchronously with analytical parent-child resolution, utilizing functions like argMax rather than the FINAL modifier to prevent costly table merges. This architectural shift allows Respan to achieve sub-second search across tens of millions of records and support customer dashboards with predictable query performance.


### [Migrating Etsy’s database sharding to Vitess](https://yomu.fyi/post/migrating-etsy-s-database-sharding-to-vitess.md)
- Company: [Etsy](https://yomu.fyi/company/etsy.md)
- Author: Ella Yarmo-Gray
- Published: Mar 19, 2026
- Type: Problem & solution

Etsy migrated its long-running MySQL shard management to Vitess vindexes after an architecture spanning about 1,000 tables and shards, 425 TB of data, and roughly 1.7 million requests per second. The legacy ORM used a single index database to map records to shards, creating a scaling bottleneck and single point of failure. To avoid moving existing data, engineers built custom SQLite lookup and hybrid vindexes, changed new shard assignments to match Vitess’s hash algorithm, and introduced routing incrementally table by table with traffic ramps, rollback, and query compatibility testing. After five years, about 2,500 pull requests, and 6,000 queries, the migration removed the index database dependency, made scaling operations take days rather than months, simplified developer access, and kept database performance largely unchanged without a mass data migration.


### [How Slack Rebuilt Notifications 📣](https://yomu.fyi/post/how-slack-rebuilt-notifications.md)
- Company: [Slack](https://yomu.fyi/company/slack.md)
- Author: Frances Coronel
- Published: Mar 19, 2026
- Type: Problem & solution

Slack redesigned its legacy notification architecture to address user confusion, conflicting desktop and mobile preference systems, and tight coupling between notification content and push delivery. The engineering team unified four separate preference models into a single framework with three channel options: All new posts, Mentions, and Mute. To ensure safe rollback and backwards compatibility without direct database mutations, Slack implemented a read-time translation strategy alongside a new desktop\_push\_enabled preference and auto-saving cross-platform React UI components. This decoupling of activity from push delivery resulted in a fivefold sustained increase in settings engagement, reduced reliance on per-channel overrides, and eliminated synchronization mismatches across mobile and desktop clients.


### [How Socialpruf built a faster, more reliable data stack by replacing Neon with Postgres managed by ClickHouse](https://yomu.fyi/post/how-socialpruf-built-a-faster-more-reliable-data-stack-by-replacing-ne.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: Mar 17, 2026
- Type: Problem & solution

Socialpruf, a social media performance tracking platform ingesting hundreds of posts per second, experienced analytics latency, connection dropouts, and high network transfer costs while running Postgres on Neon alongside ClickHouse Cloud. To resolve these issues, the team migrated their transactional system of record to NVMe-backed Postgres managed by ClickHouse, colocating transactional and analytical workloads. Although standard Postgres logical replication failed on their 0.5 TB dataset, they achieved continuous synchronization using PeerDB before conducting a production cutover. The migration eliminated connection dropouts, reduced network transfer fees, and improved overall Postgres query performance by approximately 30 percent. To manage thousands of concurrent connections generated by ingestion workers, Socialpruf implemented ClickHouse's bundled, horizontally scalable PgBouncer instances.


### [Shopify’s journey to faster breadth-first GraphQL execution](https://yomu.fyi/post/shopify-s-journey-to-faster-breadth-first-graphql-execution.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 12, 2026
- Type: Explainer

Shopify describes how high-cardinality, deeply nested GraphQL queries exposed execution overhead beyond data loading, with field resolvers assembling responses consuming much of request time. Its GraphQL Cardinal engine replaces conventional depth-first traversal with breadth-first execution, resolving each selected field once across an aggregated set of objects and reducing repeated field-level work. For large list queries, Shopify reports up to 15x faster execution, 90% less memory use, and reductions of many seconds in P50 latency. The migration included a tuned interpreter, cheaper tracer behavior, shadow verification, benchmarks, and ongoing conversion of tens of thousands of legacy resolvers, while future work includes asynchronous patterns and lower-level C bindings.


### [How Critical Manufacturing uses ClickHouse to bring real-time intelligence to the factory floor](https://yomu.fyi/post/how-critical-manufacturing-uses-clickhouse-to-bring-real-time-intellig.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: Mar 10, 2026
- Type: Problem & solution

Critical Manufacturing faced performance and scaling bottlenecks while processing millions of factory floor events using a legacy Microsoft SQL Server analytics stack. Increasing data volumes caused historical queries to take hours, straining real-time dashboards and escalating operational maintenance. The engineering team replaced this architecture by migrating analytics workloads to ClickHouse connected directly to streaming Kafka topics. They implemented an ELT pattern alongside ReplacingMergeTree engines to deduplicate late-arriving sensor records and utilized TTL policies for automated lifecycle management. The resulting platform delivers sub-second analytical queries across historical and real-time operational data, expands into OpenTelemetry observability storage, and maintains embeddings for retrieval-augmented generation workloads.


### [Innovating with MongoDB | Customer Successes, February 2026](https://yomu.fyi/post/innovating-with-mongodb-customer-successes-february-2026.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Katie Palmer
- Published: Feb 25, 2026
- Type: Problem & solution

Modern artificial intelligence startups frequently face operational bottlenecks when deploying autonomous agents and dynamic interfaces on rigid relational schemas or restricted database engines. To address these limitations, platforms including Modelence, Thesys, Emergent Labs, and Heidi integrated MongoDB Atlas as their operational data foundation. Emergent Labs eliminated recursive schema migration failures previously encountered with PostgreSQL by adopting document structures that match native JSON generation, supporting millions of deployed applications. Meanwhile, Heidi migrated from Amazon DocumentDB to utilize MongoDB Vector Search for retrieval-augmented generation pipelines under a single unified API without requiring external search databases. These architectural transitions streamlined continuous agent iterations, eliminated operational scaling downtime, and accelerated deployment cycles across diverse production environments.


### [Engineering at scale, with purpose.](https://yomu.fyi/post/engineering-at-scale-with-purpose.md)
- Company: [Thumbtack](https://yomu.fyi/company/thumbtack.md)
- Author: Thumbtack People Team
- Published: Feb 5, 2026
- Type: Explainer

Senior software engineer Brett Shouse outlines site reliability engineering initiatives and infrastructure modernization efforts underway at Thumbtack. The engineering organization is currently leading an operational project to migrate multiple self-hosted observability services onto a single, unified SaaS platform. This architectural transition consolidates application logs, distributed traces, and system metrics into one accessible view tailored for engineers, customer support staff, and company executives. Moving away from legacy self-hosted monitoring systems reduces systems administration overhead, mitigates alert fatigue, and lowers direct operational infrastructure costs. Concurrently, the reliability team tackles technical debt accumulated from rapid organizational growth by establishing structured incident response processes and automating repetitive operational toil.


### [That’s a Wrap: MongoDB’s 2025 in Review & 2026 Predictions](https://yomu.fyi/post/that-s-a-wrap-mongodb-s-2025-in-review-2026-predictions.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Kevin O'Rourke
- Published: Dec 22, 2025
- Type: Announcement

MongoDB recapped its product releases, executive transitions, and enterprise adoption milestones throughout 2025 alongside predictions for 2026. The company acquired embedding and reranking specialist Voyage AI to mitigate large language model hallucinations and launched MongoDB AMP to accelerate legacy application modernization. Vector search and full-text search capabilities were expanded from MongoDB Atlas to Community Edition and Enterprise Server, enabling on-premises and hybrid retrieval-augmented generation workloads. Enterprise implementations included pharmaceutical distributor McKesson scaling tracking infrastructure to handle 1.2 billion containers annually after replacing its monolithic legacy system. Executive predictions for 2026 emphasized human-in-the-loop verification, grounding language models with factual retrieval, and building multi-cloud resilience against outages and data sovereignty constraints.


### [From Python3.8 to Python3.10: Our Journey Through a Memory Leak](https://yomu.fyi/post/from-python3-8-to-python3-10-our-journey-through-a-memory-leak.md)
- Company: [Lyft](https://yomu.fyi/company/lyft.md)
- Author: Jay Patel
- Published: Dec 15, 2025
- Type: Incident / postmortem

During an initiative to upgrade Python services from version 3.8 to 3.10, Lyft engineers encountered severe latency spikes and timeouts in one service within their test environment. Profiling DynamoDB queries revealed that gevent thread joins were taking up to 30 seconds while pod memory consumption climbed steadily. To isolate the issue, engineers used an internal tracemalloc-based profiler triggered via SIGUSR2 signals, temporarily disabling Gunicorn preload to prevent workers from terminating prematurely on signal receipt. Memory traces pointed to botocore and an incompatibility between weakref.finalize and gevent monkey patching in urllib3 version 1.26.16, which prevented connections from returning to the pool. Downgrading urllib3 to version 1.26.15 immediately resolved both the timeouts and the memory leak before a permanent fix arrived in gevent and urllib3 updates.


### [Treating Data as Code at Two Sigma](https://yomu.fyi/post/treating-data-as-code-at-two-sigma.md)
- Company: [Two Sigma](https://yomu.fyi/company/two-sigma.md)
- Author: Emily Majewski
- Published: Nov 13, 2025
- Type: Problem & solution

Scaling research and trading platforms across thousands of data sources created bottlenecks at Two Sigma, where reliance on database snapshots and fragmented infrastructure slowed dataset delivery to data scientists. To address operational costs and architectural complexity, the data engineering team adopted software development principles by treating data as code. The organization migrated to Google BigQuery's serverless architecture, standardized SQL transformations using dbt, and defined declarative pipelines with Terraform under continuous integration workflows. Internal tooling was introduced to track directed acyclic graph dependencies, automate anomaly detection, and streamline data discovery. These shifts eliminated manual data movement routines, reduced operational overhead, and enabled formal data contracts to safeguard downstream consumers and emerging language model integrations.


### [huggingface\_hub v1.0: Five Years of Building the Foundation of Open Machine Learning](https://yomu.fyi/post/huggingface-hub-v1-0-five-years-of-building-the-foundation-of-open-mac.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Lucain Pouget, Célina Hanouti, Lysandre, Julien Chaumond
- Published: Oct 27, 2025
- Type: Announcement

The Hugging Face Hub team released huggingface\_hub v1.0, marking the library's transition into a mature foundation for open machine learning after five years of development. The major update migrates the underlying network backend to httpx and replaces the legacy hf\_transfer tool with the hf\_xet binary package to enable chunk-level file deduplication across repositories. In addition, the release introduces a redesigned Typer-based CLI using the streamlined hf command alongside explicit authentication functions and expanded inference primitives. Legacy components, including the Git-based Repository class and the older InferenceApi interface, have been removed to establish a cleaner and more maintainable architecture. Most dependent machine learning libraries remain compatible across versions, though transformers requires v0.x for version 4 and v1.x for its upcoming version 5 release.


### [Cars24 Improves Search For 300 Million Users With MongoDB Atlas](https://yomu.fyi/post/cars24-improves-search-for-300-million-users-with-mongodb-atlas.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Nick Bell
- Published: Oct 12, 2025
- Type: Problem & solution

Online car marketplace Cars24 previously operated a dual architecture where relational databases like Postgres synchronized data into separate search engines like Elasticsearch. As service offerings expanded globally for 300 million users, multiple engineering teams piping data into a single search index caused severe synchronization challenges, operational overhead, and developer friction. To simplify this infrastructure, Cars24 migrated to MongoDB Atlas hosted on Google Cloud, utilizing integrated Apache Lucene-powered MongoDB Atlas Search directly within the database layer. This architectural shift eliminated synchronization pipelines while accelerating search feature delivery through a unified API. Cars24 also replaced its ArangoDB geospatial search deployment with MongoDB Atlas, achieving robust multi-document ACID transactions across shards and cutting operational costs by 50%.


### [Innovating with MongoDB | Customer Successes, October 2025](https://yomu.fyi/post/innovating-with-mongodb-customer-successes-october-2025.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Katie Palmer
- Published: Oct 2, 2025
- Type: Announcement

Enterprise organizations are modernizing legacy monoliths, mainframes, and relational databases to improve scalability and system performance. MongoDB launched its Application Modernization Platform, an AI-powered program combining artificial intelligence tooling with migration methodologies. Using these capabilities, Wells Fargo implemented an operational data store to serve 40 percent of external vendor traffic with sub-second response times across more than 7 million transactions. CSX utilized MongoDB Atlas on Azure alongside Cluster-to-Cluster Sync to complete a cloud migration within hours while sustaining continuous operations. Additionally, Intellect Design reduced onboarding workflow times by 85 percent, and Deutsche Telekom scaled daily customer interactions from under 50,000 to approximately 1.5 million.


### [From Niche NoSQL to Enterprise Powerhouse: The Story of MongoDB's Evolution](https://yomu.fyi/post/from-niche-nosql-to-enterprise-powerhouse-the-story-of-mongodb-s-evolu.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Ashish Kumar
- Published: Sep 25, 2025
- Type: Explainer

Early perceptions categorized MongoDB alongside eventually consistent NoSQL databases, creating doubts regarding its suitability for mission-critical enterprise workloads. Over fifteen years of development, the platform addressed enterprise requirements by introducing replica sets with Raft-style consensus, native horizontal sharding, tunable read and write concerns, and multi-document ACID transactions across sharded clusters. Security capabilities were expanded through Role-Based Access Control, Field-Level Encryption, and Queryable Encryption supporting prefix, suffix, and substring searches. Current initiatives focus on decoupling compute and storage scaling, developing AI-driven tooling within the Application Modernization Platform to translate legacy relational applications to microservices, and optimizing query execution for non-denormalized data models. MongoDB Atlas now incorporates Atlas Search, Atlas Vector Search, native Stream Processing, and embedding generation powered by Voyage AI models.


### [MongoDB Community Edition to Atlas: A Migration Masterclass With BharatPE](https://yomu.fyi/post/mongodb-community-edition-to-atlas-a-migration-masterclass-with-bharat.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Nick Bell
- Published: Sep 21, 2025
- Type: Problem & solution

BharatPE, an Indian fintech processing over ₹12,000 crore monthly across 450 cities, operated a 45-terabyte self-hosted MongoDB Community Edition deployment. Managing three sharded three-node clusters resulted in uneven data distribution, high maintenance costs, scaling friction, and disaster recovery gaps under strict regulatory requirements. To modernize infrastructure, the database operations team partnered with MongoDB to execute a structured five-stage migration to MongoDB Atlas. The initiative utilized mongosync for secure data transition alongside environment mirroring and automated integrity validation scripts. Transitioning to MongoDB Atlas yielded a 40 percent improvement in query response times and provided automated failover backed by a 99.995 percent uptime SLA.


### [Modernizing Core Insurance Systems: Breaking the Batch Bottleneck](https://yomu.fyi/post/modernizing-core-insurance-systems-breaking-the-batch-bottleneck.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Vinod Bagal, Jagpreet Singh
- Published: Sep 18, 2025
- Type: Problem & solution

Migrating core insurance platforms from legacy relational databases with PL/SQL to Java and MongoDB Atlas frequently degrades batch processing performance. Like-for-like migrations can cause batch jobs to run 25 to 30 times slower due to high network round-trips, inefficient per-record operations, and under-utilized bulk capabilities. To resolve these bottlenecks, an optimization framework was developed that leverages MongoDB native bulkWrite operations, intelligent reference data prefetching, and parallel execution. An architecture combining a Spring Boot controller and a dedicated executor framework partitions workloads across threads or event processors. This framework recovered lost performance, completing previously failing batch jobs within service-level agreements and achieving 10 to 15 times better execution speeds than legacy systems in certain workloads.


### [MongoDB.local NYC 2025: Defining the Ideal Database for the AI Era](https://yomu.fyi/post/mongodb-local-nyc-2025-defining-the-ideal-database-for-the-ai-era.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Dev Ittycheria, President and CEO, MongoDB
- Published: Sep 18, 2025
- Type: Announcement

At MongoDB.local NYC, MongoDB introduced new database capabilities, AI integrations, and modernization tools designed for agentic and generative AI workloads. The release of MongoDB 8.2 arrives alongside public previews of Search and Vector Search for both MongoDB Community Edition and Enterprise Server. MongoDB also highlighted Voyage AI embedding models and rerankers to improve retrieval accuracy across raw data, metadata, and embeddings. To address the costs and constraints of legacy infrastructure, the new MongoDB Application Modernization Platform combines AI-assisted tooling and specialized expertise to migrate legacy systems. According to MongoDB, early migrations using this platform run two to three times faster, while code rewriting tasks accelerate by an order of magnitude.


[Newer posts](https://yomu.fyi/topic/migration/page/3.md) · [Older posts](https://yomu.fyi/topic/migration/page/5.md)