Tiger Data (Timescale)

How We Scaled PostgreSQL to 350 TB+

Timescale dogfooded its own Postgres extension to build a database-observability product, using hypertables, columnar compression and tiered storage to keep a 350 TB+ dataset queryable.

Published
2023-11-02
Scale
350 TB+, 10 billion new records/day
Techniques
partitioning compression tiered storage
Stack
PostgreSQL TimescaleDB

Try this stack yourself

  • Railway · Deploy something like this on Railway: spin up PostgreSQL plus your app in a few clicks. New accounts get $20 in credit (applied when you pick a paid plan).
    Railway isn't part of Tiger Data (Timescale)'s setup in this story. It's just an easy place to try the same building blocks.

Some links above are affiliate links: if you sign up, How They Scaled may earn a commission at no extra cost to you. We only list tools that fit the stack in the story.

See also

ClickHouse vs Postgres for analytics at scalePostgreSQL vs MySQL at scaleSharding vs partitioning

The best scaling write-ups, weekly

One email a week. No spam.

Subscribe

Related stories

Netflix · 2026

TimeSeries Tiered Storage Journey: Kafka/Flink Streams to Native Cassandra Cold Reads

Netflix moved older data in its multi-petabyte TimeSeries Abstraction out of hot Cassandra storage in two steps. First, Kafka and Flink streamed events to S3 as Parquet files, which a daily job compacted into small, time-sorted row groups with Bloom filters cached in EVCache (p50 ~150 ms, p99 ~500 ms). The second design skips that streaming pipeline: a ReadOnlyCompactionStrategy aligns immutable SSTables to token ranges, and a custom pluggable Memtable (Cassandra 4.1) reads directly from existing Cassandra backups in S3. It now holds over 15 PB of compressed cold data, cut disk usage on affected clusters from 80% to 7%, and improved p90 latency by ~30%.

15+ PB compressed data in cold storagedisk usage cut from 80% to 7%tiered storagecompactioncost optimizationCassandraKafkaFlink
PlanetScale · 2026

118 million queries per second on Neki

PlanetScale benchmarked Neki, its sharded Postgres, with single-shard primary-key point selects. Targeting about 200K QPS per shard, it went from 5 shards (about 1M QPS) to 50 (9.9M QPS, per-shard rate within 0.8%) to 512 shards and 480 routers, sustaining 118.5M QPS for 16 minutes over 1.22 PiB with a p99 of 6.06 ms at the router and about one error per 1.8 million queries. PlanetScale notes the run was read-only, primary-only with no replicas, and had no failover during the measured window.

118.5M QPS sustained for 16 minutes512 shards, 480 routershorizontal shardingbenchmarkinglinear scalabilityPostgreSQLNekiAWS r8g.16xlarge
Datadog · 2026

When failover isn't safe: Building high-availability PostgreSQL on Kubernetes

A gameday that injected network latency into one availability zone left several of Datadog's Kubernetes-based PostgreSQL clusters stuck: replicas lagged past Patroni's maximum_lag_on_failover, so no standby could be promoted safely. Datadog moved to a hybrid model: standbys in the leader pool take part in synchronous replication (one synchronous standby, synchronous_mode_strict, synchronous_commit=remote_apply) while read replicas stay asynchronous. pgbench showed remote_apply raised average latency by 53% and cut throughput by 34%, yet production clusters saw no measurable impact after a gradual rollout, and synchronous replication is now enabled in all of its clusters.

+53% avg write latency with remote_apply (pgbench TPC-B)-34% tps with remote_apply (pgbench TPC-B)synchronous replicationfailoverhigh availabilityPostgreSQLKubernetesPatroni
OpenAI · 2026

Scaling PostgreSQL to power 800 million ChatGPT users

OpenAI runs ChatGPT on a single-primary Postgres with nearly 50 global read replicas. Reads are offloaded, write-heavy shardable workloads move to Cosmos DB, and PgBouncer, cache leasing and multi-layer rate limits keep the primary safe.

800 million usersmillions of QPSread replicasconnection poolingcachingPostgreSQLAzurePgBouncer