domains / families / data / huggingface-community-evals

huggingface-community-evals

authentic

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.

When to use

• | Use case | Script | |---|---| | Local `inspect-ai` eval on a Hub model via inference providers | `scripts/inspect_eval_uv.py` | | Local GPU eval with `inspect-ai` using `vllm` or Transformers | `scripts/inspect_vllm_uv.py` | | Local GPU…

When not to use

• HF Jobs orchestration
• model-card PRs
• .eval_results publication
• community-evals automation
• model-card or model-index edits
• README table extraction
• Artificial Analysis imports
• PR creation

Skill page

Source