domains / families / agent-tooling / ai-agent-evaluation-benchmarking

ai-agent-evaluation-benchmarking

authentic

Autonomous AI agent benchmark evaluation register: task completion rates, planning accuracy, tool invocation precision, and cost benchmarks.

When to use

• When recording agent evaluation runs with a fixed dataset, rubric, runtime/model revision and measured outcomes.
• When comparing a candidate with a baseline under matched tasks, environment, tool permissions and cost accounting.
• When generating reproducible CSV, SQL DDL, JSON Schema, and Notion property registers for tracking compliance.

When not to use

• This register does not execute benchmarks, prove compliance or need on-chain RPC credentials. An actual authorized harness and evaluation evidence are external prerequisites.
• Scores are comparable only with matched suite/revision, eligibility, failure/timeout handling and sampling. Record counts and uncertainty, not merely a percentage.
• Define cost per solved task as total attributable run cost divided by solved tasks; zero solves means unavailable/infinite cost under an explicit policy, not zero. Token estimates alone omit provider/tool charges.

Skill page

Source