Sharding: 4 real scaling case studies
4 of the 73 engineering write-ups in our database are tagged with the technique "sharding". They come from 4 companies (Datadog, Discord, Meta and Stripe) and were published between 2017 and 2024. 2 of the 4 state concrete scale numbers, for example "1.5 PB" (Stripe). Each card below links to our short summary and the original post.
How Stripe's document databases supported 99.999% uptime with zero-downtime data migrations
Stripe's DocDB and its Data Movement Platform move data between shards without downtime.
Husky: Exactly-once ingestion and multi-tenancy at scale
A deep dive on how Husky routes, shards and ingests events exactly once across tenants.
ZippyDB: Facebook's key value store
Meta's general-purpose distributed key-value store built on RocksDB, with shard management and configurable consistency.
How Discord Indexes Billions of Messages
Discord built message search on many small Elasticsearch clusters, sharding by guild and indexing lazily through a queue.