---
title: "Scalability"
description: "379 posts about Scalability, summarised, each linking to the original."
---

# Scalability
> 379 posts about Scalability, summarised, each linking to the original.

## Articles

### [Minions: Stripe’s one-shot, end-to-end coding agents](https://yomu.fyi/post/minions-stripe-s-one-shot-end-to-end-coding-agents.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Alistair Gray
- Published: Feb 9, 2026
- Type: Explainer

Stripe’s Minions are homegrown, fully unattended coding agents designed to complete tasks in a single run, beginning with an invocation such as a Slack thread and ending with a pull request ready for human review. More than 1,000 pull requests merged weekly at Stripe are minion-produced and contain no human-written code, although engineers review them and can provide follow-up instructions. Each run uses an isolated, pre-warmed devbox and a customized fork of Block’s coding agent goose, combining agent loops with deterministic operations for version control, linting, and testing. Minions use curated MCP tools, including Stripe’s Toolshed with more than 400 tools, to retrieve internal context before and during work. Local checks and selective CI provide feedback, with no more than two CI rounds, balancing correction against token, compute, and time costs.


### [The Future of the Global Open-Source AI Ecosystem: From DeepSeek to AI+](https://yomu.fyi/post/the-future-of-the-global-open-source-ai-ecosystem-from-deepseek-to-ai.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Adina Yakefu, Irene Solaiman
- Published: Feb 3, 2026
- Type: Explainer

This third and final article in a series examines how Chinese AI organizations are shaping an open-source ecosystem after DeepSeek R1's January 2025 release. It argues that open sharing of models, papers, deployment infrastructure, and techniques has become the dominant near-term approach, with organizations ranging from Alibaba, Tencent, ByteDance, and Baidu to startups and application companies contributing at different layers. Qwen illustrates the ecosystem strategy: by mid-2025, more than 113,000 Hugging Face models used Qwen as a base, while organizations also connected models with chips, cloud platforms, toolchains, evaluation, and deployment systems. The article places this shift on infrastructure built since 2017, including the East Data, West Compute plan, and says R1 catalyzed wider use of existing compute, energy, and data capacity. It concludes that open-source models are becoming reusable, composable components in industrial workflows and localized AI systems as China moves from DeepSeek toward AI+.


### [Architectural Choices in China's Open-Source AI Ecosystem: Building Beyond DeepSeek](https://yomu.fyi/post/architectural-choices-in-china-s-open-source-ai-ecosystem-building-bey.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Adina Yakefu, Irene Solaiman
- Published: Jan 27, 2026
- Type: Explainer

Chinese open-source artificial intelligence development has shifted focus toward architectural choices, ecosystem toolchains, and domestic hardware integration following the January 2025 DeepSeek release. Leading groups have adopted Mixture-of-Experts architectures like Kimi K2 and Qwen3 to balance capability, dynamic compute allocation, and operating costs. Development expanded beyond text into multimodal domains including speech, video, and 3D systems while prioritizing smaller 0.5B to 30B distilled models under permissive Apache 2.0 licensing. Furthermore, model releases increasingly provide day-zero alignment with domestic inference and training infrastructure, including Huawei Ascend, Cambricon, and Baidu Kunlun chips. Production serving engines such as Moonshot AI's Mooncake and Baidu's FastDeploy 2.0 reflect a transition from standalone models to comprehensive system-level design.


### [Unlocking Agentic RL Training for GPT-OSS: A Practical Retrospective](https://yomu.fyi/post/unlocking-agentic-rl-training-for-gpt-oss-a-practical-retrospective.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Jason Zhu, Hejian Sang, Arup De, Rohit Jain, Yanning Chen
- Published: Jan 27, 2026
- Type: Problem & solution

Engineers evaluated the GPT-OSS model for agentic reinforcement learning across multi-step tasks such as ReTool and GSM8K using the verl framework. Initial training runs suffered from exploding gradient norms, increasing KL divergence, and stagnating rewards. Investigation revealed that non-deterministic expert routing in the Mixture of Experts architecture caused log-probability mismatches between forward passes, which falsely triggered Proximal Policy Optimization clipping. The team resolved this by substituting detached current log-probabilities for old log-probabilities during on-policy updates. To address further training instabilities and memory bottlenecks, the implementation integrated an attention-sink-aware backward pass into FlashAttention v3 and applied sequence parallelism using all-to-all communication around attention layers.


### [SkyPilot at Shopify: Multi-cloud GPUs without the pain](https://yomu.fyi/post/skypilot-at-shopify-multi-cloud-gpus-without-the-pain.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jan 26, 2026
- Type: Explainer

Shopify uses SkyPilot to run machine-learning training workloads across GPU clusters on Nebius and GCP, addressing scarce hardware, fragmented capacity, and provider-specific interfaces. Persistent Kubernetes clusters remain managed by Shopify, while SkyPilot launches jobs and a company plugin routes requests, validates labels, injects configuration, and applies organizational policies. H200 workloads go to Nebius for InfiniBand-equipped distributed training, while L4 and CPU-only jobs go to GCP; labels also connect jobs to cost ownership, Kueue quotas, priority classes, and development-environment rules. The plugin automatically configures InfiniBand access, shared caches, and temporary storage, reducing manual setup and speeding repeated workloads. The resulting YAML-based interface preserves direct resource control while hiding cloud-selection decisions, giving teams common workflows as providers, prices, and GPU availability change.


### [Scaling long-running autonomous coding](https://yomu.fyi/post/scaling-long-running-autonomous-coding.md)
- Company: [Cursor](https://yomu.fyi/company/cursor.md)
- Author: Wilson Lin
- Published: Jan 14, 2026
- Type: Explainer

The post reports experiments running coding agents autonomously for weeks, with hundreds concurrently contributing to projects that typically take human teams months. It describes initial dynamic coordination through shared files, locks, and optimistic concurrency control, then explains why flat peer structures caused bottlenecks, fragility, risk aversion, and stalled progress. The successful design separates recursive planners from task-focused workers, uses a judge between cycles, and lets workers push changes to a shared branch with minimal conflicts. In tests, agents built a browser with more than 1 million lines across 1,000 files, migrated Solid to React over three weeks with +266K/-193K edits, and delivered a 25x faster Rust video renderer; other projects remain underway. The authors say GPT-5.2 models perform better on extended autonomous work, role-specific models and carefully tuned prompts matter, and moderate structure outperforms both flat peer coordination and elaborate controls, though drift and scheduling remain unsolved.


### [That’s a Wrap: MongoDB’s 2025 in Review & 2026 Predictions](https://yomu.fyi/post/that-s-a-wrap-mongodb-s-2025-in-review-2026-predictions.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Kevin O'Rourke
- Published: Dec 22, 2025
- Type: Announcement

MongoDB recapped its product releases, executive transitions, and enterprise adoption milestones throughout 2025 alongside predictions for 2026. The company acquired embedding and reranking specialist Voyage AI to mitigate large language model hallucinations and launched MongoDB AMP to accelerate legacy application modernization. Vector search and full-text search capabilities were expanded from MongoDB Atlas to Community Edition and Enterprise Server, enabling on-premises and hybrid retrieval-augmented generation workloads. Enterprise implementations included pharmaceutical distributor McKesson scaling tracking infrastructure to handle 1.2 billion containers annually after replacing its monolithic legacy system. Executive predictions for 2026 emphasized human-in-the-loop verification, grounding language models with factual retrieval, and building multi-cloud resilience against outages and data sovereignty constraints.


### [Token-count-based Batching: Faster, Cheaper Embedding Inference for Queries](https://yomu.fyi/post/token-count-based-batching-faster-cheaper-embedding-inference-for-quer.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Chengcheng Pei, Yuan Lin
- Published: Dec 18, 2025
- Type: Problem & solution

Serving embedding models for short search queries often suffers from poor GPU efficiency because traffic is spiky and memory-bound. Traditional time-window and request-count batching strategies lead to inconsistent GPU utilization, while tensor padding wastes compute on empty tokens. To resolve these bottlenecks, Voyage AI implemented token-count-based batching paired with padding removal in inference engines like vLLM. The architecture uses Redis with Lua scripts to atomically aggregate pending requests until reaching an optimal hardware saturation token threshold. Across production deployments, this approach achieved a 50% reduction in GPU inference latency with 3X fewer GPUs and improved throughput by up to 8×.


### [Building production-ready Stripe subscriptions using Kiro powers](https://yomu.fyi/post/building-production-ready-stripe-subscriptions-using-kiro-powers.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Kalyani Koppisetti
- Published: Dec 4, 2025
- Type: Tutorial

Kiro's Stripe power is presented as a way to build a production-ready SaaS subscription flow while avoiding the fragmented output common with generic coding assistants. Using a natural-language specification for three monthly tiers and a three-month one-time option, Kiro creates a React frontend, Node.js backend API, CDN-hostable static assets, and server-side Stripe Checkout Session creation. The integration includes Stripe-specific steering files, MCP servers, and hooks, while the webhook handler preserves the raw request body, verifies HMAC signatures with stripe.webhooks.constructEvent(), rejects invalid events with HTTP 400, and routes lifecycle events such as checkout completion and subscription cancellation. The post concludes that this provides a secure, scalable foundation while leaving teams to customize business logic, including access provisioning and deprovisioning, instead of building payment infrastructure from scratch.


### [Continuous batching from first principles](https://yomu.fyi/post/continuous-batching-from-first-principles.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Rémi Ouazan Reboul, Arthur Zucker, Luc Georges
- Published: Nov 25, 2025
- Type: Explainer

Large language model serving requires running expensive next-token generation across multiple concurrent user requests. Traditional batching approaches introduce severe padding inefficiencies when mixing variable-length prompts and different generation phases, especially under static shape constraints like CUDA graphs. Continuous batching resolves these inefficiencies by combining key-value caching, chunked prefill, ragged batching, and dynamic request scheduling. Ragged batching eliminates the traditional batch axis by concatenating token sequences into a single tensor and using boolean attention masks to isolate independent sequences. By dynamically removing completed prompts and packing decoding tokens alongside chunked prefill tokens up to a hardware memory budget, serving systems maintain high hardware utilization and throughput.


### [How we prepare Shopify for BFCM](https://yomu.fyi/post/how-we-prepare-shopify-for-bfcm.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 20, 2025
- Type: Explainer

Shopify’s BFCM readiness program addresses the challenge of operating critical commerce infrastructure at traffic levels informed by 2024 records, while preparing APIs with no prior holiday-traffic history. Starting in March, teams combined capacity planning on Google Cloud, an infrastructure roadmap, risk assessments, chaos-engineering Game Days, and a Resiliency Matrix covering failure modes, recovery objectives, runbooks, and on-call coverage. Genghis load tests and Toxiproxy network faults ran across three GCP regions, while five April–October scale tests exercised browsing, checkout, analytics, webhooks, regional failover, and cascading failures; testing reached 146 million requests per minute and a p99 target of 200 million. The work exposed and addressed Kafka partition, memory, timeout, routing, queueing, and failover issues, while the operational plan adds monitoring, 24/7 incident coverage, merchant communications, live tuning, and a post-BFCM review.


### [Building world-class product search at Shopify: Where C++ excellence meets ML innovation](https://yomu.fyi/post/building-world-class-product-search-at-shopify-where-c-excellence-meet.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 12, 2025
- Type: Problem & solution

Shopify describes a product-search stack built for multi-tenant commerce, where real-time indexing, retrieval, ranking, experimentation, and merchant controls must operate at global scale. Commerce ranking combines classical information retrieval with transformers, neural rankers, LightGBM, Catboost, and signals such as relevance, purchase popularity, brand trust, and navigational intent, while supporting privacy, multi-language, and accessibility requirements. RankFlow, a Python-like domain-specific language, lets data scientists change ranking logic without C++ expertise, while TurboDSL compiles and executes it with type safety, vectorized operations, cache optimization, predictable latency, and efficient memory use. The team first shipped SimScorerDSL in C++, then built TurboDSL without changing the DSL syntax, enabling continued experimentation during the transition. TurboDSL achieved a 48% speedup in ranking feature computation, and automated pull-request analysis compares component timings, statistical significance within a ±2% tolerance, and side-by-side results to detect performance or accuracy regressions.


### [Grab's Mac Cloud Exit supercharges macOS CI/CD](https://yomu.fyi/post/grab-s-mac-cloud-exit-supercharges-macos-ci-cd.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Madushan Gamage
- Published: Nov 6, 2025
- Type: Problem & solution

Grab moved its macOS CI/CD infrastructure from a US cloud vendor to a colocation cluster in Malaysia, closer to its Git servers in Southeast Asia. The change followed growth from one office Mac Pro to more than 250 Mac minis, rising demand, job queues, and cloud costs that made ownership or colocation more attractive at scale. Grab selected bare-metal hardware after comparing cloud, colocation, and on-premises options, then migrated progressively while monitoring failure rates, performance, and reliability; the later infrastructure also uses zero-touch provisioning to reduce manual setup as the cluster grows. The migration produced projected savings of more than $2.4 million over three years and performance improvements of 20–40%, while the team chose allocated-kilowatt power commitments and retained bare metal because macOS virtualization’s performance or stability trade-offs did not justify it.


### [Cars24 Improves Search For 300 Million Users With MongoDB Atlas](https://yomu.fyi/post/cars24-improves-search-for-300-million-users-with-mongodb-atlas.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Nick Bell
- Published: Oct 12, 2025
- Type: Problem & solution

Online car marketplace Cars24 previously operated a dual architecture where relational databases like Postgres synchronized data into separate search engines like Elasticsearch. As service offerings expanded globally for 300 million users, multiple engineering teams piping data into a single search index caused severe synchronization challenges, operational overhead, and developer friction. To simplify this infrastructure, Cars24 migrated to MongoDB Atlas hosted on Google Cloud, utilizing integrated Apache Lucene-powered MongoDB Atlas Search directly within the database layer. This architectural shift eliminated synchronization pipelines while accelerating search feature delivery through a unified API. Cars24 also replaced its ArangoDB geospatial search deployment with MongoDB Atlas, achieving robust multi-document ACID transactions across shards and cutting operational costs by 50%.


### [The Cost of Not Knowing MongoDB, Part 3: appV6R0 to appV6R4](https://yomu.fyi/post/the-cost-of-not-knowing-mongodb-part-3-appv6r0-to-appv6r4.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Artur Costa
- Published: Oct 9, 2025
- Type: Benchmark

To alleviate a disk throughput bottleneck on a MongoDB server, this optimization pass focuses on shrinking document sizes across application revisions appV6R0 through appV6R4. The implementation replaces an array of items with a dynamic schema document, using date segments such as days or combined months and days as field names to store status totals. Redundant date components already encoded in the binary \_id field are omitted from internal document keys. Bulk upsert operations rely on updateOne with the $inc operator to update matching date keys or create documents when missing. Aggregation pipelines process reports using the $objectToArray and $reduce operations to calculate totals across date intervals while preserving a single index on \_id.


### [How AI Agents are Evolving Shopify's Product Taxonomy at Scale (2025) - Shopify](https://yomu.fyi/post/how-ai-agents-are-evolving-shopify-s-product-taxonomy-at-scale-2025-sh.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 9, 2025
- Type: Explainer

Shopify’s product classification system processes over tens of millions of predictions daily, while its taxonomy contains more than 10,000 categories and 2,000 attributes that must adapt as commerce changes. Manual curation created bottlenecks, expertise gaps, and inconsistencies that could reduce discoverability, search quality, filtering, and classification quality. The proposed AI multi-agent system combines structural analysis with product-driven analysis grounded in merchant titles, descriptions, and categories, then uses intelligent synthesis, equivalence detection, and quality assurance to recommend taxonomy improvements. Agents can search and validate hierarchical relationships, while human experts retain oversight; Shopify tested the method in Electronics > Communications > Telephony against its previous manual expansion approach and presents it as a proactive, scalable way to improve taxonomy consistency before gaps affect merchants and customers.


### [Innovating with MongoDB | Customer Successes, October 2025](https://yomu.fyi/post/innovating-with-mongodb-customer-successes-october-2025.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Katie Palmer
- Published: Oct 2, 2025
- Type: Announcement

Enterprise organizations are modernizing legacy monoliths, mainframes, and relational databases to improve scalability and system performance. MongoDB launched its Application Modernization Platform, an AI-powered program combining artificial intelligence tooling with migration methodologies. Using these capabilities, Wells Fargo implemented an operational data store to serve 40 percent of external vendor traffic with sub-second response times across more than 7 million transactions. CSX utilized MongoDB Atlas on Azure alongside Cluster-to-Cluster Sync to complete a cloud migration within hours while sustaining continuous operations. Additionally, Intellect Design reduced onboarding workflow times by 85 percent, and Deutsche Telekom scaled daily customer interactions from under 50,000 to approximately 1.5 million.


### [Carrying Complexity, Delivering Agility](https://yomu.fyi/post/carrying-complexity-delivering-agility.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Akshat Vig, Ashish Kumar
- Published: Sep 25, 2025
- Type: Explainer

MongoDB centers its engineering architecture around resilience, intelligence, and simplicity to minimize developer cognitive and operational burdens when building distributed applications. Security is enforced through dedicated clusters in isolated virtual private networks alongside Queryable Encryption, which allows equality and range queries on ciphertext without decryption keys ever leaving the client. High availability is built on replica sets across independent availability zones and multi-cloud topologies, utilizing consensus mechanisms that commit writes only after majority acknowledgment in the active term. Operational friction in artificial intelligence workloads is addressed by integrating vector search directly into the core query engine, eliminating brittle extract-transform-load pipelines and separate vector databases.


### [From Niche NoSQL to Enterprise Powerhouse: The Story of MongoDB's Evolution](https://yomu.fyi/post/from-niche-nosql-to-enterprise-powerhouse-the-story-of-mongodb-s-evolu.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Ashish Kumar
- Published: Sep 25, 2025
- Type: Explainer

Early perceptions categorized MongoDB alongside eventually consistent NoSQL databases, creating doubts regarding its suitability for mission-critical enterprise workloads. Over fifteen years of development, the platform addressed enterprise requirements by introducing replica sets with Raft-style consensus, native horizontal sharding, tunable read and write concerns, and multi-document ACID transactions across sharded clusters. Security capabilities were expanded through Role-Based Access Control, Field-Level Encryption, and Queryable Encryption supporting prefix, suffix, and substring searches. Current initiatives focus on decoupling compute and storage scaling, developing AI-driven tooling within the Application Modernization Platform to translate legacy relational applications to microservices, and optimizing query execution for non-denormalized data models. MongoDB Atlas now incorporates Atlas Search, Atlas Vector Search, native Stream Processing, and embedding generation powered by Voyage AI models.


### [MongoDB Community Edition to Atlas: A Migration Masterclass With BharatPE](https://yomu.fyi/post/mongodb-community-edition-to-atlas-a-migration-masterclass-with-bharat.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Nick Bell
- Published: Sep 21, 2025
- Type: Problem & solution

BharatPE, an Indian fintech processing over ₹12,000 crore monthly across 450 cities, operated a 45-terabyte self-hosted MongoDB Community Edition deployment. Managing three sharded three-node clusters resulted in uneven data distribution, high maintenance costs, scaling friction, and disaster recovery gaps under strict regulatory requirements. To modernize infrastructure, the database operations team partnered with MongoDB to execute a structured five-stage migration to MongoDB Atlas. The initiative utilized mongosync for secure data transition alongside environment mirroring and automated integrity validation scripts. Transitioning to MongoDB Atlas yielded a 40 percent improvement in query response times and provided automated failover backed by a 99.995 percent uptime SLA.


[Newer posts](https://yomu.fyi/topic/scalability/page/10.md) · [Older posts](https://yomu.fyi/topic/scalability/page/12.md)