How to Build a Diffusion Language Model

Diffusion models have revolutionized image generation, but applying them to text was long an open problem. This guide from the Kuleshov Group explains how masked diffusion works—training a bidirectional transformer to fill in randomly masked tokens, then generating text by iterative unmasking. It covers the probabilistic foundations, block diffusion for variable-length output, architectural choices, and techniques like distillation and post-training. By 2026, diffusion LLMs like Mercury 2, Gemma Diffusion, and Nemotron Diffusion are competitive with autoregressive models, offering faster generation, error correction, and bidirectional context.