domains / families / research / 08-eval-harness

08-eval-harness

separate

Run and score the agent p-hacking benchmark. Composes prompt cells across research framing and significance pressure, drives an agent through them, and scores each run into a P-Hacking Intensity index decomposed into selection, search breadth, estimate inflation, inference gap, pre-registration departure and disclosure. Use when benchmarking whether an AI agent p-hacks, comparing models or tool stacks on statistical integrity, scoring a single agent analysis run, or designing an evaluation of statistical guardrails.

When to use

• benchmarking whether an AI agent p-hacks
• comparing models or tool stacks on statistical integrity
• scoring a single agent analysis run
• designing an evaluation of statistical guardrails

When not to use

• scoring method-selection correctness
• scoring diagnostic completeness
• scoring hallucination
• scoring refusal of unidentifiable designs

Skill page

Source