domains / families / science / hugging-face-community-evals

hugging-face-community-evals

authentic

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate.

When to use

• `inspect-ai` with local inference
• `lighteval` with local inference
• choosing between `vllm`, Hugging Face Transformers, and `accelerate`
• smoke tests, task selection, and backend fallback strategy
• Hugging Face Jobs orchestration
• model-card or `model-index` edits
• README table extraction
• Artificial Analysis imports

When not to use

• Use this skill only when the task clearly matches its upstream product or API scope.
• Verify commands, API behavior, pricing, quotas, credentials, and deployment effects against current official documentation before making changes.
• Do not treat generated examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.

Skill page

Source