AGENTS.md outperforms skills in our agent evals
Vercel just proved the 'dumb' approach wins: a static 8KB markdown file achieved 100% pass rates where agent skills maxed out at 79%.
Why it matters
Vercel's AGENTS.md framework challenges the assumption that sophisticated retrieval systems beat simple context injection for AI coding agents. This has immediate implications for how framework authors and tool builders should architect agent knowledge systems.
The key facts
10 to knowAGENTS.md achieved 100% pass rate vs. 79% for skills with explicit instructions
Skills were never invoked in 56% of eval cases without explicit instruction
Compressed 8KB docs index (80% reduction from 40KB original)
Skill with explicit instructions achieved 79% pass rate (+26pp vs baseline)
Baseline (no docs) achieved 53% pass rate
AGENTS.md showed perfect scores across Build (100%), Lint (100%), and Test (100%) categories
Skills approach introduced performance degradation on some metrics (58% vs 63% on tests)
Available via: npx @next/codemod@canary agents-md
Research conducted by Jude Gao at Vercel
Tested against Next.js 16 APIs not in model training data ('use cache', connection(), forbidden(), etc.)
Go to the source
Vercel Blogvercel.com
Publisher excerpt: We expected to be the solution for teaching coding agents framework-specific knowledge. After building evals focused on Next.js 16 APIs, we found something unexpected.skills A compressed 8KB docs index embedded directly in achieved a 100% pass rate, while skills maxed out at 79% even with explicit…
