FrontierSeptember 14, 2026via MarkTechPost
Sakana AI Researchers Introduce PC-ALM, a Layer-Local Alternative to Backpropagation That Trains 1000-Layer Networks
Why it matters
A novel training algorithm that recovers backprop-quality gradients through layer-local computation could reshape how deep networks are trained — especially relevant for edge deployment, neuromorphic hardware, and scaling laws that depend on depth.
Key signals
- PC-ALM (Predictive Coding with Augmented Lagrangian Multipliers) is a layer-local alternative to backpropagation
- Recovers exact backprop gradients in linear networks; matches BP across widths 8-128 and depths up to 1000
- Inference budget: T = 2L (2 iterations per layer)
- Gradient cosine similarity improves from 0.604 to 0.909 on reference cell
- Trains 1000-layer residual MLPs within ~2 percentage points of backprop on MNIST
- MIT-licensed JAX implementation released
- Authors: Jeffrey Seely and Julian Gould (Sakana AI)
- Addresses biological plausibility and local-learning constraints in deep neural networks
- PC-ALM (Augmented Lagrangian Predictive Coding) achieves layer-local updates without sacrificing gradient quality
- Matches backprop performance across 8-128 layer depths at inference budget T=2L
- Gradient cosine similarity improves from 0.604 to 0.909 in reference cell
- Successfully trains 1000-layer residual MLPs within ~2 points of backprop on MNIST
- MIT-licensed JAX code released publicly
- Researchers: Jeffrey Seely and Julian Gould (Sakana AI)
- Solves the credit-assignment problem for biologically plausible learning
The hook
Sakana AI cracks a 40-year problem: training 1000-layer networks without backpropagation. PC-ALM matches backprop's accuracy at a fraction of the compute cost.
Sakana AI researchers Jeffrey Seely and Julian Gould introduce Augmented Lagrangian Predictive Coding (PC-ALM), a local-learning alternative to backpropagation. By attaching a Lagrange multiplier to each layer constraint, PC-ALM keeps predictive coding's layer-local updates while recovering exact ba…