What happened after 2,000 people tried to hack my AI assistant
2,000 hackers tried to break an AI assistant. Here's what actually worked.
Why it matters
Real-world adversarial testing of AI systems reveals practical security gaps that matter to anyone deploying agents in production. A crowdsourced red-teaming exercise exposes the gap between theoretical safety and deployed reality.
The key facts
10 to know2,000 people participated in adversarial testing
Published by Simon Willison (Datasette creator, AI safety researcher)
Focuses on practical attack vectors and failure modes
Empirical data on AI assistant robustness under real adversarial pressure
Relevant to AI safety governance and production deployment risk
2,000 people participated in adversarial testing/hacking attempt
Published by Simon Willison (prominent AI/web developer, known for AI safety commentary)
Large-scale empirical data on AI assistant robustness and attack vectors
Real-world security findings from deployed AI system
Relevant to AI governance, safety testing, and responsible deployment
Go to the source
Simon Willisonsimonwillison.net