domains / families / agent-tooling / agent-evaluation-reporting

agent-evaluation-reporting

authentic

Use when summarizing agent evaluations where autonomous, assisted, failed, timed-out, or invalid outcomes must remain distinct and comparable.

When to use

• Use when reporting benchmark, regression, pilot, or production evaluation runs for an AI agent.
• Use when autonomous and human-assisted completions appear in the same result set.
• Use when failures, timeouts, infrastructure-invalid runs, retries, or partial results affect the denominator.
• Use when comparing two agents, prompts, harnesses, or releases and deciding whether the comparison is valid.

When not to use

• This skill structures and interprets supplied evaluation evidence; it does not validate the evaluator or recreate missing run records.
• Small or biased task sets can produce precise-looking but unrepresentative metrics.
• Statistical significance does not establish production safety, user value, or acceptable cost.
• Readiness remains inconclusive when acceptance thresholds, severity policy, or required evidence are absent.

Skill page

Source