Researchers train AI model that hits near-full performance with just 12.5 percent of its experts
3/4 of the experts. That's what researchers just pruned from a mixture-of-experts model without breaking it.
Why it matters
Allen Institute and UC Berkeley have demonstrated a novel MoE architecture (EMO) that maintains near-full performance while using only 12.5% of experts by organizing specialization around content domains rather than token types. This breakthrough could make MoE models viable for memory-constrained and edge deployments, reshaping inference economics.
The key facts
5 to knowEMO model strips 87.5% of experts with only ~1 percentage point performance loss
Domain-specialized experts instead of token-type specialization (novel architecture approach)
Researchers: Allen Institute for AI and UC Berkeley
Use case: Memory-constrained settings and practical MoE deployment
Published: May 16, 2026
Go to the source
The Decoderthe-decoder.com
Publisher excerpt: Researchers at the Allen Institute for AI and UC Berkeley have built EMO, a mixture-of-experts model whose experts specialize in content domains instead of word types. That lets you strip out three-quarters of the experts while losing only about one percentage point of performance, a step that…