domains / families / research / advanced-evaluation

advanced-evaluation

authentic

This skill should be used for advanced LLM evaluation: LLM-as-judge systems, direct scoring, pairwise comparison, rubric calibration, evaluator bias mitigation, confidence scoring, and automated quality assessment.

When to use

• Building LLM-as-judge systems
• Comparing multiple model responses
• Establishing consistent quality standards
• Debugging inconsistent evaluation results
• Designing A/B tests for prompt or model changes
• Creating rubrics for LLM or hybrid judges
• Analyzing correlation between automated and human judgments
• Mitigating evaluator bias

When not to use

• General deterministic checks
• Regression suites
• Production quality gates
• Outcome metrics
• Autonomous loop governance
• Locked rubrics
• Rollback
• PR approval boundaries

Skill page

Source