Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models
Apple researchers challenge the autoregressive monopoly: diffusion models match LLM accuracy at a fraction of the compute cost.
Why it matters
A fundamental shift in how language models work could reshape inference economics and energy consumption. If diffusion language models achieve parity with autoregressive models on accuracy while solving the sequential-dependency bottleneck, practitioners will need to reconsider which architectures to deploy and train.
The key facts
6 to knowResearch from Apple's Machine Learning lab
Compares Diffusion Language Models (DLMs) vs Autoregressive Language Models (ARMs)
DLMs address the low arithmetic intensity problem in next-token prediction
Scope: document processing and code generation tasks
Published August 7, 2026
Core finding: DLMs emerge as viable alternative to sequential dependency model
Go to the source
Apple Machine Learningmachinelearning.apple.com
Publisher excerpt: Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks, including document processing and code generation. Autoregressive Language Models (ARMs), which generate tokens sequentially conditioned on all previous tokens, have…