Hardware reliability: 2 real scaling case studies
2 of the 73 engineering write-ups in our database are tagged with the technique "hardware reliability". They come from 1 company (Meta) and were published between 2024 and 2026. Often combined with distributed training (2) and networking (2). 1 of the 2 state concrete scale numbers, for example "gigawatt-scale supercluster" (Meta). Each card below links to our short summary and the original post.
Inside Prometheus: Meta's Journey to a Gigawatt-Scale Supercluster
Meta turned its New Albany, Ohio data center site from web traffic into Prometheus, a gigawatt-scale AI supercluster spanning multiple interconnected buildings that behaves as one machine. It runs NVL72 racks (Catalina on GB200 with 800G of scale-out per GPU, Clemente on GB300 in a single rack) on a dedicated Ethernet back-end network for GPU-to-GPU traffic. Reliability work pushed pre-training goodput above 85% and mean time between interruptions past 54K GPU hours, and the cluster is growing toward roughly 1 million GPUs.
How Meta trains large language models at scale
Meta covers the hardware, networking and reliability work needed to train LLMs across very large GPU clusters.