Datadog architecture: how Datadog scaled (6 case studies)
Datadog's Husky is its third-generation event store: a distributed, schemaless column store that keeps event data on object storage, written in Go, with exactly-once ingestion and multi-tenant sharding built on Kafka and FoundationDB. Datadog also redesigned its timeseries indexing in Rust on RocksDB to handle a large jump in ingested data.
We track 6 public engineering write-ups by Datadog, published between 2022 and 2026. Techniques that come up more than once: multi-tenancy (2). Stack mentioned most: Go (2) and PostgreSQL (2). 3 of the 6 state concrete scale numbers, for example "+53% avg write latency with remote_apply (pgbench TPC-B)". Each card below links to our short summary and the original post.
Tech used in Datadog's scaling write-ups
Datadog scaling timeline: every story, summarized
- 2022-05-17
Introducing Husky, Datadog's third-generation event store
Datadog's Husky is a distributed, schemaless column store that keeps event data on object storage.
- 2023-02-22
Husky: Exactly-once ingestion and multi-tenancy at scale
A deep dive on how Husky routes, shards and ingests events exactly once across tenants.
- 2024-06-28
Timeseries indexing at scale
Datadog redesigned its timeseries indexing to handle a large jump in ingested data.
- 2025-06-17
Breaking up a monolith: How we're unwinding a shared database at scale
Datadog is splitting its large shared Postgres database into team-owned instances in three phases. Logical separation moves tables into about 30 private domain schemas with explicit ACLs, using query samples and interviews with 30+ teams. Access reduction routes services through per-team PG Proxies so ACLs can be narrowed safely, and physical separation replicates each schema to a managed OrgStore instance, moves reads first and then writes with under five seconds of downtime, and replicates back for rollback. Two workloads have fully migrated and more than 70 dead tables were deleted along the way.
- 2025-11-18
Scaling real-time file monitoring with eBPF: How we filtered billions of kernel events per minute
Datadog's eBPF-based file integrity monitoring sees more than 10 billion file events per minute, and at about 5 KB each, shipping them all would mean terabytes per second. Agent-side rules cut what goes upstream to roughly one million events per minute, and moving evaluation into the kernel stops the ring buffer from overflowing: static approvers compiled from detection rules let matching events through, while runtime discarders stored in LRU eBPF maps drop values no rule can match. Together they pre-filter up to 94% of events in the kernel with no dropped events.
- 2026-06-04
When failover isn't safe: Building high-availability PostgreSQL on Kubernetes
A gameday that injected network latency into one availability zone left several of Datadog's Kubernetes-based PostgreSQL clusters stuck: replicas lagged past Patroni's maximum_lag_on_failover, so no standby could be promoted safely. Datadog moved to a hybrid model: standbys in the leader pool take part in synchronous replication (one synchronous standby, synchronous_mode_strict, synchronous_commit=remote_apply) while read replicas stay asynchronous. pgbench showed remote_apply raised average latency by 53% and cut throughput by 34%, yet production clusters saw no measurable impact after a gradual rollout, and synchronous replication is now enabled in all of its clusters.
FAQ
What is Datadog Husky?
Husky is Datadog's third-generation event store, introduced in 2022: a distributed, schemaless column store that keeps event data on object storage, with storage separated from compute.
How does Husky ingest data exactly once?
Datadog's 2023 deep dive explains how Husky routes, shards and ingests events exactly once across tenants, using Go, Kafka and FoundationDB.
How did Datadog scale timeseries indexing?
Datadog's 2024 post describes redesigning its timeseries indexing, with Rust and RocksDB, to handle a large jump in ingested data.