How teams scaled Cosmos DB: 2 real case studies
2 of the 73 engineering write-ups in our database mention Cosmos DB in their stack. They come from 1 company (OpenAI) and were published in 2026. Techniques that show up most often alongside Cosmos DB: connection pooling (2) and rate limiting (2). 2 of the 2 state concrete scale numbers, for example "70 million+ requests/sec" (OpenAI). Each card below links to our short summary and the original post.
Rapidly scaling online storage to serve over 1 billion ChatGPT users
OpenAI's Habitat storage platform grew from a Python client library over Cosmos DB into a central service handling 70M+ requests per second. The team ran many low-concurrency Python processes to keep asyncio delay down, switched aiohttp connection reuse from LIFO to FIFO to stop a metastable failure, used Envoy for connection fan-in, kept a deliberately simple NoSQL API, then rewrote the service in Rust.
Scaling PostgreSQL to power 800 million ChatGPT users
OpenAI runs ChatGPT on a single-primary Postgres with nearly 50 global read replicas. Reads are offloaded, write-heavy shardable workloads move to Cosmos DB, and PgBouncer, cache leasing and multi-layer rate limits keep the primary safe.