How does this skill prevent Goodhart's Law?

By keeping the evaluation criteria static and separate from the optimization loop, the system cannot 'learn' to trick the tests without providing actual functional improvements.

Does Eval Harness require human intervention?

Yes. While it automates many regression checks, it enforces mandatory human gates for breaking changes, high-risk security updates, and instances where automated auditors disagree.

What kind of tests does Eval Harness run?

It executes specialized benchmark suites for prompt quality, skill structure, expertise accuracy, cognitive frame application, and cross-lingual integration.

What is a 'frozen' evaluation harness?

A frozen harness uses fixed, manually maintained benchmarks that do not self-improve, ensuring metrics remain objective and are not gamed or overfitted by the AI model.

Can I use Eval Harness to test multi-lingual outputs?

Yes, it includes specific suites to evaluate cross-lingual integration quality, such as Turkish evidential markers or Japanese hierarchical registers.

Eval Harness (Frozen Evaluation)

Name: Eval Harness (Frozen Evaluation)
Author: DNYoussef

byDNYoussef

•

Security & Testing

Gates and validates self-improvement cycles through frozen benchmarks and regression testing to prevent AI quality degradation.

The Eval Harness is a critical safety and quality assurance layer for the Context Cascade system, providing a strictly 'frozen' environment to evaluate cognitive frame application and cross-lingual integration. By decoupling evaluation metrics from the self-improvement loop, it prevents Goodhart’s Law—where an AI optimizes for the metric rather than the actual outcome—ensuring that system enhancements lead to real-world performance gains. This skill manages complex benchmark suites for prompt and skill generation, expertise file precision, and multi-lingual consistency while enforcing mandatory human approval gates for high-risk architectural changes.

Key Features

01Falsifiable pattern validation for domain expertise and analytical tasks

02Automated regression testing for prompt forge and skill generation logic

03Multi-stage human approval gates for high-risk or breaking changes

046 GitHub stars

05Cognitive frame application and cross-lingual integration quality gates

06Frozen benchmark suites for objective, non-gaming performance tracking

Use Cases

01Enforcing manual review for security-sensitive or core logic modifications

02Preventing quality regressions when generating new autonomous AI skills

03Measuring the linguistic accuracy of cross-lingual cognitive mapping

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add dnyoussef/context-cascade tooling

For use in Claude.ai and ChatGPT

Download Skill