A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models
A single neuron. That's all it takes to break safety alignment across 7 major LLMs — and researchers just proved it.
Why it matters
Apple researchers demonstrate a critical vulnerability in how LLMs are safeguarded: safety mechanisms are mechanistically brittle and can be defeated at the neuron level without training. This finding has immediate implications for how AI companies should architect alignment and raises governance questions about model robustness standards.
The key facts
6 to knowSafety alignment operates through two distinct systems: refusal neurons and concept neurons
Vulnerability demonstrated across 7 models spanning 1.7B to 70B parameters
Both refusal suppression and harmful content amplification achieved without training or prompt engineering
Affects two model families
Published by Apple Machine Learning Research
Mechanistic vulnerability in alignment architecture
Go to the source
Apple Machine Learningmachinelearning.apple.com
Publisher excerpt: Safety alignment in language models operates through two mechanistically distinct systems: refusal neurons that gate whether harmful knowledge is expressed, and concept neurons that encode the harmful knowledge itself. By targeting a single neuron in each system, we demonstrate both directions of…