domains / families / software-engineering / llm-eval-harness

llm-eval-harness

separate

Tests/benchmarks a third-party LLM endpoint (OpenAI- or Anthropic-compatible): availability, fidelity, speed, concurrency, protocol compliance, quality regression. Use before hand-rolling a curl loop, when onboarding a provider, or debugging "system prompt 不生效" / a tok/s claim — 测评/压测一个模型或渠道. Not for TTS/voice-clone supplier eval (use the audio skill).

When to use

• onboarding a provider
• debugging system prompt issues
• verifying tok/s claims
• testing model availability
• checking protocol compliance
• quality regression testing
• concurrency and stability testing
• request fidelity verification

When not to use

• TTS/voice-clone supplier eval

Skill page

Source