What is Eval-Driven Development (EDD)?

EDD treats evaluations as unit tests for AI, defining expected behavior before implementation to ensure accuracy and reliability throughout the development lifecycle.

Does it support human intervention?

Yes, it includes a Human Grader type specifically designed for tasks requiring manual review, such as security-sensitive changes or complex UI interactions.

How does Eval Harness track reliability?

It uses metrics like pass@k (success in k attempts) and pass^k (consistent success across all attempts) to provide a quantitative score for AI performance.

Can I use my own test scripts with Eval Harness?

Yes, the skill supports code-based graders that can execute shell commands, grep patterns, or standard test runners like npm test to verify results deterministically.

Eval Harness

Name: Eval Harness
Author: xu-xiang

byxu-xiang

•

Security & Testing

Implements Eval-Driven Development (EDD) within Claude Code to ensure code quality through formal evaluation frameworks.

Eval Harness brings the principles of Eval-Driven Development (EDD) to Claude Code, allowing developers to treat evaluations as unit tests for AI-generated code. It provides a structured workflow to define expected behaviors before coding, track regressions during development, and measure reliability using metrics like pass@k. By supporting code-based, model-based, and human graders, it ensures that every code change is validated against specific criteria, making AI-assisted development more predictable and production-ready.

Key Features

01Multi-modal grading (Code-based, Model-based, and Human review)

0261 GitHub stars

03Automated reporting and version-controlled evaluation logs

04Capability and Regression evaluation templates

05Integrated Eval-Driven Development (EDD) workflow

06Reliability tracking with pass@k and pass^k metrics

Use Cases

01Automating regression testing to prevent AI-generated bugs in existing codebases

02Defining success criteria for a new feature before implementation

03Measuring the reliability of Claude's output across multiple attempts

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add xu-xiang/everything-claude-code-zh eval-harness

For use in Claude.ai and ChatGPT

Download Skill