How does this prevent LLMs from 'gaming' tests?

Traditional tests can be 'gamed' if an LLM changes a threshold (like changing a timeout from 50ms to 100ms) to make a test pass. Spec tests require an intent statement; if the code violates the intent, Claude marks it as a failure regardless of the raw output.

Where should I store my spec tests?

Spec tests should be stored in the 'specs/tests/' directory. Each file should use Markdown format and include frontmatter to define its target source files.

What is an intent-based spec test?

An intent-based spec test is a natural language specification that explains why a test exists. Claude acts as a judge to determine if the code satisfies the underlying business need, rather than just checking if a technical value matches.

Do I need a separate API key for the test runner?

No, the included test runner uses the 'claude -p' command, which utilizes your existing Claude subscription and environment settings.

Intent-Based Spec Tests

Name: Intent-Based Spec Tests
Author: ianphil

byianphil

0•

Security & Testing

Implements natural language specification testing using Claude as a judge to ensure code aligns with business intent and user requirements.

This skill enables a robust, intent-based testing framework that uses Claude as an evaluator to verify that your code fulfills specific business logic rather than just passing arbitrary technical assertions. By mandating intent statements for every test case, it creates a 'cheat-proof' TDD environment where Claude prevents code from 'gaming' the tests, ensuring that UX requirements, performance targets, and functional goals are strictly met. It is ideal for developers using AI-driven workflows who need to bridge the gap between natural language requirements and technical implementation.

Key Features

01LLM-as-judge evaluation for natural language specifications

02Multi-target support to validate logic across multiple files simultaneously

03Markdown-based test format for high readability and documentation

04Full TDD workflow support: Plan, Spec, Implement, and Verify

05Mandatory intent statements to capture the 'why' behind every test

060 GitHub stars

Use Cases

01Guiding AI code generation to ensure it meets requirements without cutting corners

02Enforcing non-negotiable UX requirements like latency and responsiveness

03Validating complex business logic that is difficult to express in traditional unit tests

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add ianphil/my-skills spec-tests

For use in Claude.ai and ChatGPT

Download Skill