How teams scaled PyTorch: 2 real case studies
2 of the 73 engineering write-ups in our database mention PyTorch in their stack. They come from 1 company (Meta) and were published between 2024 and 2026. Techniques that show up most often alongside PyTorch: distributed training (2). It's often paired with GPUs (2). 1 of the 2 state concrete scale numbers, for example "2.5x model complexity" (Meta). Each card below links to our short summary and the original post.
Meta's Generative Ads Model (GEM): Scaling the Central Model and Compute Engine for Frontier Ads AI
Meta replaced separate per-surface ad prediction systems with GEM, one central recommendation foundation model trained across all of its ad surfaces. A new LLM-style architecture (Kunlun) showed LLM-like scaling with 2.5x more model complexity, while custom low-precision kernels and 5D parallelism doubled end-to-end training efficiency to 20-25% MFU at 4x the training FLOPs across thousands of GPUs. Post-training transfer (LoopFM) and elastic pre-computation (Solaris) pass GEM's gains to real-time serving models without extra latency.
How Meta trains large language models at scale
Meta covers the hardware, networking and reliability work needed to train LLMs across very large GPU clusters.