Loading post…
GithubBrian CelenzaProblem & solution
Building Git infrastructure for agent-scale development
Summary
GitHub is rebuilding its Git infrastructure to handle agentic development, where agents push millions of commits daily and total activity doubled to 473.3 billion events per month. The current Spokes architecture couples durability with scale by storing full repository copies on local disks and using a quorum for writes, making writes slower as read replicas are added. Key challenges include commit turnaround latency per agent, write throughput up 4.9x year over year, merges contending on a single reference, and CI fan-out multiplying each push into thousands of reads. The new design separates storage from compute, using Azure Blob Storage for durability and lightweight workers for reads, shrinks the critical path by coordinating only reference updates, and moves maintenance off the serving path. Internal benchmarks show up to 35 times higher write throughput with independently scaling read capacity, while preserving existing workflows, branch protections, and audit controls.
Context
GitHub’s current Git infrastructure, Spokes, stores full repository copies on local disks and uses a three-phase commit quorum for consistency. This couples durability with scale, making writes slower as read replicas are added. Agentic development drives massive concurrent writes and reads, with pushes growing 4.9x year over year and CI runs 4x. The busiest repository saw a billion requests in August 2026.
Approach / What changed
The new architecture separates storage from compute: authoritative repository data lives in Azure Blob Storage for durability, while lightweight compute workers cache data to serve reads. Coordination is minimized by only requiring agreement on reference updates, while object storage, validation, and secret scanning happen in parallel. Maintenance tasks like compaction and garbage collection are moved to separate workers that operate directly against durable storage, not on serving hosts. This allows reads to scale independently without adding durable copies, and compute workers can be added or removed based on demand.
Takeaways
GitHub’s total Git activity doubled from 218.2 billion to 473.3 billion events per month between September 2025 and August 2026, with pushes growing 4.9x year over year.
The current Spokes architecture couples durability with scale: adding read replicas makes writes slower because every replica participates in every write via a three-phase commit quorum.
The new design separates storage from compute, using Azure Blob Storage for durability and lightweight workers for reads, achieving up to 35 times higher write throughput in internal benchmarks.
FAQ
Why does adding read replicas slow down writes in the current architecture?
In Spokes, every replica participates in every write via a three-phase commit quorum. A push is only as fast as the slowest replica, so adding replicas to absorb read load adds overhead to writes.
How does the new architecture separate storage from compute?
Authoritative repository data lives in Azure Blob Storage for durability, while lightweight compute workers cache data to serve requests. This lets read capacity scale independently without adding durable copies, and compute workers can be added or removed based on demand.
What throughput improvements does the new architecture show?
Internal benchmarks show up to 35 times higher write throughput, with read capacity scaling independently to meet demand.
Source and freshness
- Source
- github.blog
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.