FrontierThe story, in brief
Tilde Research Introduces Aurora: A Leverage-Aware Optimizer That Fixes a Hidden Neuron Death Problem in Muon
Muon has a hidden killer. Tilde Research just proved it—and shipped Aurora to fix it.
The KeyNews take
Why it matters
A structural flaw in Muon, a widely-adopted training optimizer, silently kills neurons during pretraining. Aurora's leverage-aware fix could reshape how teams approach LLM efficiency and training cost.
The key facts
9 to knowAurora addresses neuron death problem in Muon optimizer
1.1B parameter pretraining experiment included
New state-of-the-art result achieved
Muon is widely-used in neural network training
Leverage-aware optimization approach
Aurora optimizer targets 'neuron death' flaw in Muon
Flaw permanently kills MLP neurons during training
Leverage-aware optimizer architecture
Research from Tilde Research
Go to the source
MarkTechPostmarktechpost.com
Publisher excerpt: Researchers at Tilde Research have released Aurora, a new optimizer for training neural networks that addresses a structural flaw in the widely-used Muon optimizer. The flaw quietly kills off a significant fraction of MLP neurons during training and keeps them permanently dead. Aurora comes with a…