Partitioning: 3 real scaling case studies
3 of the 73 engineering write-ups in our database are tagged with the technique "partitioning". They come from 3 companies (Discord, LinkedIn and Tiger Data (Timescale)) and were published between 2016 and 2023. 2 of the 3 state concrete scale numbers, for example "350 TB+" (Tiger Data (Timescale)). Each card below links to our short summary and the original post.
How We Scaled PostgreSQL to 350 TB+
Timescale dogfooded its own Postgres extension to build a database-observability product, using hypertables, columnar compression and tiered storage to keep a 350 TB+ dataset queryable.
How Discord Stores Billions of Messages
Early story of Discord outgrowing MongoDB and moving messages to Cassandra with a channel+time-bucket partition key.
FollowFeed: LinkedIn's Feed Made Faster and Smarter
LinkedIn moved its feed to a fan-out-on-read system that keeps member timelines in memory and on RocksDB.