domains / families / business / ai-evals

ai-evals

separate

Help users build robust infrastructure for measuring, monitoring, and iterating on AI product performance using human, code-based, and LLM-as-a-judge methodologies.

When to use

• Identify failure modes
• Select eval methods
• Build gold sets
• Operationalize evaluations
• AI evaluation strategy
• Error analysis on real traces
• Curating reference datasets
• Integrating evaluations into CI/CD

When not to use

• Relying on vibe checks
• Obsessing over prompt engineering

Skill page

Source