GitHub

Building Git infrastructure for agent-scale development

GitHub explains why agent workloads strain Spokes, which keeps five durable copies of each repo on fileserver disks, so every added read replica slows writes. The new design coordinates only the ref update, moves compaction and GC to separate workers, and separates storage from compute: Azure Blob Storage holds the authoritative data while lightweight caching workers serve reads.

Published
2026-10-06
Scale
473.3 billion Git events/month, 7.38 billion commits in Sept 2026, 3.35 billion pushes/month, up to 35x write throughput (internal benchmarks)
Techniques
control plane separation read replicas caching workload isolation
Stack
Git Spokes Azure Blob Storage

See also

GitHub architectureRead replicas vs sharding

The best scaling write-ups, weekly

One email a week. No spam.

Subscribe

Related stories

GitHub · 2026

The bridge to agent-scale Git infrastructure

GitHub has copied 99.99999% of its Git repositories to Azure, where most now keep two of their five replicas and Azure serves about half of Git traffic. The busiest remaining repositories were blocked by a ~25ms cross-region round trip, because Spokes held a repository-wide lock for each write while waiting on the slowest healthy replica. Scoping the quorum to individual Git references lets writes to different branches proceed in parallel (3-4x higher throughput in internal testing), and a virtual write replica that streams writes into Azure Blob Storage lets stateless read replicas scale out on demand.

99.99999% of repositories copied to AzureAzure serves about half of Git trafficper-reference quorumstorage/compute separationcloud migrationGitSpokesAzure
OpenAI · 2026

Scaling PostgreSQL to power 800 million ChatGPT users

OpenAI runs ChatGPT on a single-primary Postgres with nearly 50 global read replicas. Reads are offloaded, write-heavy shardable workloads move to Cosmos DB, and PgBouncer, cache leasing and multi-layer rate limits keep the primary safe.

800 million usersmillions of QPSread replicasconnection poolingcachingPostgreSQLAzurePgBouncer
Cloudflare · 2026

Under the hood, off the hot path: how D1 scales to millions of databases

How Cloudflare's D1 keeps query execution at the edge while a separate control plane manages millions of SQLite databases.

millions of databasescontrol plane separationmulti-tenancyedge computingSQLiteDurable ObjectsWorkers
Canva · 2026

Session revocations at scale

Canva keeps 12 hours of session revocations in memory on every gateway, but hundreds of pods each loading over a million revocations from MySQL on deploy caused a stampede on the database. A worker now packs revocations into sorted 16-byte records in 30-minute chunks on S3, using conditional PUTs for optimistic concurrency and ZooKeeper leader election to avoid conflicts, and reaches over 2,000 revocations per second. Gateways binary-search the downloaded bytes directly and poll with conditional GETs, which shrank the cache by 87.5% and cut the revocation database down to two read replicas kept for redundancy.

hundreds of millions of usershundreds of thousands of session checks/seccachingobject storageoptimistic concurrency controlMySQLAmazon S3ZooKeeper