Implements a formal evaluation framework for Claude Code sessions based on Eval-Driven Development (EDD) principles.
The Eval Harness skill brings rigorous Eval-Driven Development (EDD) to Claude Code, allowing developers to treat evaluations as unit tests for AI-generated code. It enables the definition of success criteria before implementation, tracks capability improvements and regression risks through automated grading, and provides reliability metrics like pass@k. By standardizing how Claude's performance is measured across sessions, this skill ensures that complex AI-driven tasks are completed with high confidence and verifiable results.
Key Features
01Integrated evaluation storage and history within the project structure
02Reliability tracking using pass@k and pass^k metrics
03Standardized multi-stage workflow from definition to reporting
0461 GitHub stars
05Automated capability and regression testing frameworks
06Support for code-based, model-based, and human graders
Use Cases
01Protecting existing functionality from regressions during refactoring or iterative development
02Measuring the reliability of complex logic by running multiple pass@k iterations
03Defining clear success criteria for a new feature before Claude begins implementation
