Huggingface Community Evals

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware.

by waybarrios

The skill

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.

This skill's licence does not allow redistribution, so the store does not show its text. Read it at the source.

Use this skill

Connect your agent to the Skill Store MCP server and ask for it by ref: waybarrios/huggingface-community-evals-38791558883.

Markdown version

Details

Publisher: waybarrios
Category: Software Development › Testing & QA
Updated: 2026-10-19

References

Related skills

Huggingface Best Find and compare recommended Hugging Face models for a task using benchmarks, model size, and dev…
Huggingface Llm Trainer Train or fine-tune language and vision models using TRL (Transformer Reinforcement Learning) or U…
Huggingface Local Models Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm.
Playwright E2E Write and stabilise end-to-end browser tests
Ab Testing When the user wants to plan, design, or implement an A/B test or experiment, or build a growth ex…