Anthropic Adds Plugin Evals to Claude Code: 6 Grader Types, a No-Plugin Baseline, and a CI Gate for Skills
Anthropic just shipped eval infrastructure for Claude Code plugins—a no-plugin baseline, six grader types, and a CI gate. Plugin developers can now measure whether their skills actually work.
Why it matters
Claude Code plugin developers gain measurement and validation tools that were previously unavailable. This reduces friction for shipping reliable plugins and raises the bar for plugin quality across the ecosystem.
The key facts
11 to knowNew claude plugin eval command for plugin testing
6 grader types for evaluation
No-plugin baseline comparison included
CI gate for skills validation
Measures: skill triggering, performance, improvement over baseline
Answers measurement gaps plugin developers faced
New 'claude plugin eval' command runs plugins against realistic prompts
6 grader types included for measuring skill behavior
No-plugin baseline enables A/B comparison of skill impact
CI gate integration for skill validation before deployment
Addresses three previously unmeasurable questions: does skill trigger, does it work, does it improve the outcome
Go to the source
MarkTechPostmarktechpost.com
Publisher excerpt: Anthropic has published a new plugin evals workflow for Claude Code. The claude plugin eval command runs a plugin against realistic prompts, grades what Claude produced, and compares the result with a run where the plugin is not loaded. It answers 3 questions plugin developers could not previously…