Scaling Properties of Continuous Diffusion Spoken Language Models
Apple's continuous diffusion speech models scale like large language models—and they might finally close the gap with text.
Why it matters
Apple research shows spoken language models using continuous diffusion exhibit predictable scaling laws, challenging the assumption that speech-only models require fundamentally different training approaches than text models. This could reshape how teams approach multimodal and speech AI development.
The key facts
11 to knowSpeech-only models have lagged text and text-speech models in performance
Discrete autoregressive SLMs show high computational and data demands
Continuous diffusion (CD) SLM approach tested as alternative to discretization bottleneck
New phoneme Jensen-Shannon divergence (pJSD) metric introduced for SLM evaluation
CD SLMs exhibit scaling laws for validation loss and pJSD metrics
Published by Apple Machine Learning Research on Jul 6, 2026
Continuous diffusion SLMs exhibit scaling laws for validation loss and phoneme Jensen-Shannon divergence (pJSD)
Discrete autoregressive SLMs create computational bottlenecks that limit scaling efficiency
Novel phoneme Jensen-Shannon divergence metric introduced for quantifying SLM linguistic quality
Research indicates CD approach mirrors AR scaling behavior while avoiding discretization penalties
Published by Apple Machine Learning Research team
Go to the source
Apple Machine Learningmachinelearning.apple.com
Publisher excerpt: Speech-only spoken language models (SLMs) lag behind text and text-speech models in performance, with recent discrete autoregressive (AR) SLMs indicating significant computational and data demands to match text models. Since discretizing continuous speech for AR creates bottlenecks, we explore…