ToolsSeptember 11, 2026via MarkTechPost
Anthropic Adds Plugin Evals to Claude Code: 6 Grader Types, a No-Plugin Baseline, and a CI Gate for Skills
Why it matters
Claude Code plugin developers gain measurement and validation tools that were previously unavailable. This reduces friction for shipping reliable plugins and raises the bar for plugin quality across the ecosystem.
Key signals
- New claude plugin eval command for plugin testing
- 6 grader types for evaluation
- No-plugin baseline comparison included
- CI gate for skills validation
- Measures: skill triggering, performance, improvement over baseline
- Answers measurement gaps plugin developers faced
- New 'claude plugin eval' command runs plugins against realistic prompts
- 6 grader types included for measuring skill behavior
- No-plugin baseline enables A/B comparison of skill impact
- CI gate integration for skill validation before deployment
- Addresses three previously unmeasurable questions: does skill trigger, does it work, does it improve the outcome
The hook
Anthropic just shipped eval infrastructure for Claude Code plugins—a no-plugin baseline, six grader types, and a CI gate. Plugin developers can now measure whether their skills actually work.
Anthropic has published a new plugin evals workflow for Claude Code. The claude plugin eval command runs a plugin against realistic prompts, grades what Claude produced, and compares the result with a run where the plugin is not loaded. It answers 3 questions plugin developers could not previously m…