Why does this skill enforce a two-iteration cap?

The cap prevents over-optimization and ensures that persistent failures are recognized as legitimate model limitations rather than fixable configuration errors.

Can I run the retests in parallel?

Yes, the skill utilizes helper scripts that support parallel execution (e.g., --parallel 3) to significantly speed up the retesting phase.

What kind of fixes does this skill propose?

It focuses on the smallest possible changes, such as adjusting skill descriptions, prompts, permissions, or test parameters, rather than broad code rewrites.

What is the primary purpose of the Skill Eval Optimizer?

It automates the process of identifying why a Claude skill failed its loading evaluation and helps apply minimal, targeted fixes to resolve the issue.

Which files does this skill need to function?

It requires access to evaluation results, a steering guide for fix strategies, and the opencode_skill_loading_eval_dataset.jsonl file.

Skill Eval Optimizer

Name: Skill Eval Optimizer
Author: chandima

bychandima

•

Security & Testing

Optimizes and triages failed skill-loading evaluations by applying targeted fixes and executing strict retest cycles.

Skill Eval Optimizer is a specialized workflow tool designed to manage the lifecycle of failed skill evaluations within the OpenCode ecosystem. It automates the process of listing failed cases, analyzing them against a steering guide, and proposing minimal, surgical fixes to prompts, permissions, or descriptions. By enforcing a strict two-iteration cap, it ensures that developers focus on high-impact improvements while identifying legitimate model limitations. This skill is essential for teams maintaining complex AI skill libraries who need a standardized, reproducible way to improve evaluation pass rates without over-engineering solutions.

Key Features

011 GitHub stars

02Strict iteration enforcement to prevent infinite optimization loops

03Detailed summary reporting of PASS/FAIL counts and legitimate failures

04Automated triage and listing of failed evaluation cases from test results

05Targeted retesting of specific failed IDs to reduce execution overhead

06Guided remediation strategies using a standardized steering document

Use Cases

01Iteratively improving skill metadata and prompts based on real-world eval data

02Documenting edge cases and model limitations during large-scale skill deployments

03Debugging and fixing failed skill-loading tests in a CI/CD environment

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add chandima/opencode-config skill-evals-optimize

For use in Claude.ai and ChatGPT

Download Skill