Does this skill provide automated fixes?

It generates a structured report identifying the specific test name, the reason for the ambiguity, and a proposed fix to either relax the test or clarify the specification.

What is an ambiguous test interpretation?

An ambiguous interpretation occurs when a test enforces a specific behavior, such as a specific output order or string formatting, that was never explicitly mandated in the requirements document.

When should I use the checkpoint-ambiguity-review skill?

Use it when you encounter snapshot evaluation failures that don't seem to violate explicit rules, or when auditing a new set of tests for potential brittle assertions.

How does this skill help with code erosion?

It identifies where tests are failing due to 'interpretive drift' in SlopCodeBench, helping you distinguish between actual logic errors and tests that are simply too restrictive.

Test Ambiguity Auditor

Name: Test Ambiguity Auditor
Author: SprocketLab

bySprocketLab

•

Security & Testing

Identifies and resolves test assertions that enforce arbitrary interpretations rather than explicit requirements from specifications.

The Test Ambiguity Auditor is a specialized skill designed to align software specifications with their corresponding test suites. By analyzing checkpoint documents and Python tests, it detects 'interpretive drift'—instances where tests penalize valid code because of unstated assumptions regarding output ordering, tie-breaking, formatting, or boundary behaviors. This tool is essential for developers using SlopCodeBench or iterative development cycles, ensuring that test failures represent actual requirement violations rather than accidental implementation details.

Key Features

01Detection of unstated assumptions like output ordering, casing, and rounding

02Integration with SlopCodeBench for snapshot evaluation verification

03Automated mapping of test assertions to explicit specification clauses

04Structured reporting with clear rationale and proposed fixes for each ambiguity

057 GitHub stars

06Identification of boundary behavior and error handling inconsistencies

Use Cases

01Auditing test suites to ensure they don't penalize valid alternative implementations

02Debugging snapshot evaluation failures caused by interpretive issues rather than bugs

03Refining technical requirements and test cases during iterative specification development

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add sprocketlab/slop-code-bench checkpoint-ambiguity-review

For use in Claude.ai and ChatGPT

Download Skill