Meta

Meta's Generative Ads Model (GEM): Scaling the Central Model and Compute Engine for Frontier Ads AI

Meta replaced separate per-surface ad prediction systems with GEM, one central recommendation foundation model trained across all of its ad surfaces. A new LLM-style architecture (Kunlun) showed LLM-like scaling with 2.5x more model complexity, while custom low-precision kernels and 5D parallelism doubled end-to-end training efficiency to 20-25% MFU at 4x the training FLOPs across thousands of GPUs. Post-training transfer (LoopFM) and elastic pre-computation (Solaris) pass GEM's gains to real-time serving models without extra latency.

Published
2026-10-13
Scale
2.5x model complexity, 4x total training FLOPs, 20-25% MFU (training efficiency doubled), 3.6 billion daily users across Meta apps
Techniques
recommendation systems distributed training knowledge distillation model consolidation
Stack
GPUs PyTorch custom low-precision kernels

See also

Meta architecture

The best scaling write-ups, weekly

One email a week. No spam.

Subscribe

Related stories

Meta · 2024

How Meta trains large language models at scale

Meta covers the hardware, networking and reliability work needed to train LLMs across very large GPU clusters.

distributed traininghardware reliabilitynetworkingGPUsPyTorch
Meta · 2026

Inside Prometheus: Meta's Journey to a Gigawatt-Scale Supercluster

Meta turned its New Albany, Ohio data center site from web traffic into Prometheus, a gigawatt-scale AI supercluster spanning multiple interconnected buildings that behaves as one machine. It runs NVL72 racks (Catalina on GB200 with 800G of scale-out per GPU, Clemente on GB300 in a single rack) on a dedicated Ethernet back-end network for GPU-to-GPU traffic. Reliability work pushed pre-training goodput above 85% and mean time between interruptions past 54K GPU hours, and the cluster is growing toward roughly 1 million GPUs.

gigawatt-scale superclusterhundreds of thousands of GPUs today, ~1 million plannedGPU clustersdistributed traininghardware reliabilityNVIDIA GB200 NVL72NVIDIA GB300 NVL72Ethernet back-end network
Shopify · 2026

Cross-store product clustering across millions of stores

Shopify groups the same real-world product sold by different merchants under one Universal Product Identifier. Embedding nearest-neighbor search finds plausible pairs across billions of listings, then three scorers of rising cost judge them: a CPU GBDT (L1) over trillions of pairs, a fine-tuned multilingual E5 cross-encoder (L2), and an image-aware reranker (L3) fed from prioritized GPU queues. A bounded union-find splits the graph into single-machine components, and HAC with a learned GBDT merge rule builds high-precision clusters daily, with an LLM judge validated against human labels supplying training data and cached scores reused between builds.

billions of product listings across millions of storestrillions of candidate pairs scored by L1embedding retrievaltiered model cascadebounded union-find graph partitioningGBDTMultilingual E5 cross-encoderLLM judge