domains / families / agent-tooling / agent-evals

agent-evals

combined

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality. Build automated evaluation suites for AI agents using golden datasets,

When to use

• Shipping new agent features or changing prompts
• Adding CI gates for agent quality and safety
• Building regression suites for tool-calling agents
• Measuring LLM output quality at scale
• Validating RAG retrieval accuracy

When not to use

• Guidance executes against real environments: confirm target, blast radius, and rollback plan before applying anything.
• Never deploy to production without explicit approval. Docs-only import: upstream scripts and templates not bundled.

Skill page

Conflict

The sickn33 copy is a mirror of the BagelHole skill (it even records source_repo: BagelHole/DevOps-Security-Agent-Skills), but it adds a distinct 'Evaluation contract' section with constraints the original lacks: templates are not bundled executable tests, you must supply the agent adapter and fixtures, approve provider cost and data transmission, keep fixtures synthetic, sandbox tool effects, inspect tool/network traces rather than only response words, and define thresholds and human review before evaluating. It also adds a prerequisite for pytest-timeout and slightly tightens the unit-eval JSON check. Because the mirror contains material steps and constraints the original does not, crowning the original would hide a real alternative; the combined instruction should include both the original procedure and the mirror's contract.