Zalando
Client-Side Load Balancing at a Million Requests Per Second
Zalando moved the internal fan-out of its Product Read API off the shared Skipper ingress into an in-process JVM load balancer that rebuilds the same xxHash64 consistent-hash ring. They added N-ring fade-in so new pods warm only the products they will keep, bounded load based on occupancy (Little's Law) weighted by latency, and a capped ring walk. Pods ran more evenly, the autoscaler target rose from 50% to 65% CPU, and the fleet shrank by over 25%.
- Published
- 2026-06-23
- Scale
- 1 million+ requests/sec moved off shared ingress, up to 100 downstream calls per batch, Skipper pods cut from 50+ to 8, $1,000+/day saved
- Techniques
- load balancing consistent hashing caching fan-out load shedding
- Stack
- Java Kubernetes Skipper DynamoDB
Try this stack yourself
- Better Stack · Monitor uptime and logs with Better Stack, and get alerted fast if a migration or traffic spike goes sideways on your own system.
Better Stack isn't part of Zalando's setup in this story. It's a tool for watching your own systems while you try ideas like these.
Some links above are affiliate links: if you sign up, How They Scaled may earn a commission at no extra cost to you. We only list tools that fit the stack in the story.