GitLab

Scaling Git for the agentic software factory

GitLab is building next-gen source control for thousands of concurrent AI agents working on large repositories. With conventional Git, a single agent invocation can mean 5 to 10 GB of data transfer and more than 30 seconds of setup just to answer one question. Next-gen SCM keeps the Git protocol but runs stateless compute that scales horizontally against shared object storage, including S3-compatible storage customers control, and lets agents request only the files and history they need instead of cloning whole repos. In early internal tests, workloads ran up to 50x faster with up to 1,000x less network traffic, which GitLab calls peak results it is still validating. It also notes CI is the next bottleneck: GitLab.com customers created 40% more pipelines and pushed 50% more code over the past year.

Published
2026-10-13
Scale
single agent invocation can need 5-10 GB of transfer and 30+ s of setup, up to 50x faster, 1,000x less network traffic in early internal tests, 40% more CI/CD pipelines and 50% more code pushes on GitLab.com in a year
Techniques
stateless compute storage/compute separation partial fetch agent-native interfaces
Stack
Git Object storage S3-compatible storage

See also

GitLab architecture

The best scaling write-ups, weekly

One email a week. No spam.

Open subscribe page

Related stories

OpenAI · 2026

Scaling Git at OpenAI

OpenAI built Pando, a Git-compatible backend for its monorepo after Codex pushed concurrent branches, merges and fetches past the old system (reliability sometimes below 90%). Pando stores each push as an immutable pack in blob storage and records per-branch ref versions in a horizontally scalable metadata store, giving serializability per ref and causal consistency across the repo so unrelated branches do not contend. Instead of building unique packs per client, it reuses stored packs and serves them via packfile-uri through regional Express caches (over 99% hit rate), with LSM-style background compaction so fetches plan in logarithmic time. Deployed first as a read/write-through proxy, then cut over with a Git CLI wrapper; it now powers all monorepo development, cut median push latency by nearly 5x, and can deliver nearly a petabyte of Git data per day (57+ TB in a single hour).

nearly a petabyte of Git data/daymedian push latency cut nearly 5x (~80%)per-ref causal consistencyreusable packfilesstorage/compute separationGitPandoRust
GitHub · 2026

The bridge to agent-scale Git infrastructure

GitHub has copied 99.99999% of its Git repositories to Azure, where most now keep two of their five replicas and Azure serves about half of Git traffic. The busiest remaining repositories were blocked by a ~25ms cross-region round trip, because Spokes held a repository-wide lock for each write while waiting on the slowest healthy replica. Scoping the quorum to individual Git references lets writes to different branches proceed in parallel (3-4x higher throughput in internal testing), and a virtual write replica that streams writes into Azure Blob Storage lets stateless read replicas scale out on demand.

99.99999% of repositories copied to AzureAzure serves about half of Git trafficper-reference quorumstorage/compute separationcloud migrationGitSpokesAzure
Datadog · 2022

Introducing Husky, Datadog's third-generation event store

Datadog's Husky is a distributed, schemaless column store that keeps event data on object storage.

columnar storagestorage/compute separationmulti-tenancyGoObject storage
GitHub · 2026

Building Git infrastructure for agent-scale development

GitHub explains why agent workloads strain Spokes, which keeps five durable copies of each repo on fileserver disks, so every added read replica slows writes. The new design coordinates only the ref update, moves compaction and GC to separate workers, and separates storage from compute: Azure Blob Storage holds the authoritative data while lightweight caching workers serve reads.

473.3 billion Git events/month7.38 billion commits in Sept 2026control plane separationread replicascachingGitSpokesAzure Blob Storage