domains / families / agent-tooling / eval

eval

combined

AI Agent Skill unit testing framework. A framework-agnostic toolkit for discovering, scaffolding, selecting, evaluating, and reporting on AI skills. Use this skill to assess skill quality before production, compare candidate skills on the same benchmark, enforce quality gates in CI/CD, and generate human-readable evaluation reports. Eval and improve a skill against golden cases — run the target skill blind in a fresh, context-free subagent on each example input, grade the artifact against the expected outcome, and let the gaps drive the edits. Use when the user wants to test/eval/improve/harden a skill, says "this skill keeps producing X / keeps missing Y", or hands a skill plus example input→expected-output pairs. Pairs with [write-skills](../write-skills/SKILL.md) (the authoring principles every fix obeys).

When to use

• **Before deploying a new skill to production** — run `eval` to verify it meets your quality gate.
• **When choosing between multiple candidate skills** — run `select` to rank them on the same benchmark.
• **When a skill is upgraded** — run `report diff` to detect regressions.
• **In CI/CD** — use `--exit-on-fail` to block merges that degrade skill quality.
• **When bootstrapping a new skill** — run `create` to generate a ready-to-fill skeleton.

When not to use

• agent-level evaluation
• trajectory-level evaluation

Skill page

Conflict

The two copies describe fundamentally different evaluation methodologies. The first (dzhng/skills) is a qualitative, agent-driven process: it uses fresh, context-free subagents to run a skill blind on golden cases, then uses a separate judge agent to grade the artifact against a 'bar' and 'smells' to drive iterative skill edits. The second (LeoYeAI/openclaw-master-skills) is a quantitative, CLI-based framework: it uses a command-line tool (`eval-skills`) to run benchmarks, score results with specific scorers (e.g., `exact`), and generate reports for CI/CD gates. They do not overlap in procedure; one is a manual/agent workflow for improvement, the other is an automated testing framework for quality assurance.