domains / families / data / write-code-eval

write-code-eval

separate

Write code evaluators for known failure modes with objective rules. Use when code can check the rule from a trace, with or without a reference answer. Use `write-judge-prompt` when the rule requires interpretation.

When to use

• Write code evaluators
• Check objective rules from a trace
• Implement checks in the project's language
• Test known passes and failures
• Check invalid output structure
• Check missing or forbidden text
• Check citation not in retrieved documents
• Check bad tool call

When not to use

• Rules that require interpretation
• Use write-judge-prompt

Skill page

Source