Does this skill provide automated fixes?

It generates a structured report identifying the specific test name, the reason for the ambiguity, and a proposed fix to either relax the test or clarify the specification.

What is an ambiguous test interpretation?

An ambiguous interpretation occurs when a test enforces a specific behavior, such as a specific output order or string formatting, that was never explicitly mandated in the requirements document.

When should I use the checkpoint-ambiguity-review skill?

Use it when you encounter snapshot evaluation failures that don't seem to violate explicit rules, or when auditing a new set of tests for potential brittle assertions.

How does this skill help with code erosion?

It identifies where tests are failing due to 'interpretive drift' in SlopCodeBench, helping you distinguish between actual logic errors and tests that are simply too restrictive.

Test Ambiguity Auditor

Name: Test Ambiguity Auditor
Author: SprocketLab

bySprocketLab

•

Security & Testing

Identifies and resolves test assertions that enforce arbitrary interpretations rather than explicit requirements from specifications.

About

The Test Ambiguity Auditor is a specialized skill designed to align software specifications with their corresponding test suites. By analyzing checkpoint documents and Python tests, it detects 'interpretive drift'—instances where tests penalize valid code because of unstated assumptions regarding output ordering, tie-breaking, formatting, or boundary behaviors. This tool is essential for developers using SlopCodeBench or iterative development cycles, ensuring that test failures represent actual requirement violations rather than accidental implementation details.

Key Features

Detection of unstated assumptions like output ordering, casing, and rounding
Integration with SlopCodeBench for snapshot evaluation verification
Automated mapping of test assertions to explicit specification clauses
Structured reporting with clear rationale and proposed fixes for each ambiguity
7 GitHub stars
Identification of boundary behavior and error handling inconsistencies

Use Cases

Auditing test suites to ensure they don't penalize valid alternative implementations
Debugging snapshot evaluation failures caused by interpretive issues rather than bugs
Refining technical requirements and test cases during iterative specification development

About

Key Features

Detection of unstated assumptions like output ordering, casing, and rounding
Integration with SlopCodeBench for snapshot evaluation verification
Automated mapping of test assertions to explicit specification clauses
Structured reporting with clear rationale and proposed fixes for each ambiguity
7 GitHub stars
Identification of boundary behavior and error handling inconsistencies

Use Cases

Auditing test suites to ensure they don't penalize valid alternative implementations
Debugging snapshot evaluation failures caused by interpretive issues rather than bugs
Refining technical requirements and test cases during iterative specification development