How Discord Automates ScyllaDB Clusters at Scale
Discord's 7-person persistence team replaced fragile Python and bash scripts with the Scylla Control Plane, a Rust framework of idempotent tasks with declared preconditions, composed into YAML workflows. Jobs batch nodes by availability zone with concurrency limits to protect quorum, store their state in SQLite so they resume after interruptions, and send a webhook alert on unrecoverable errors. Operations like standing up shadow clusters and rolling upgrades went from a day and a half of babysitting to under two hours.
- Published
- 2026-05-08
- Scale
- dozens of clusters, hundreds of database nodes, 7-person persistence team, 36-hour cluster operations cut to under 2 hours
- Techniques
- database operations automation idempotent tasks zone-aware rollouts resumable jobs
- Stack
- ScyllaDB Rust YAML SQLite