Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
7.6
Rating
0
Installs
AI & LLM
Category
Excellent skill documentation for evaluating code generation models. The description clearly explains what the skill does (multi-benchmark evaluation with pass@k metrics) and when to use it. The SKILL.md provides comprehensive task knowledge with four detailed workflows covering standard benchmarking, multi-language evaluation, instruction-tuned models, and model comparison. Each workflow includes checklists, complete commands, and concrete examples. The structure is well-organized with quick start, workflows, troubleshooting, and reference sections. The skill demonstrates strong novelty as coordinating 15+ benchmarks across 18 languages with proper pass@k estimation would require significant CLI work and domain expertise. Minor room for improvement: could be slightly more concise in some sections, and the workflow checklists could be more actionable. Overall, this is a production-ready skill that would significantly reduce the cost and complexity of benchmarking code models compared to manual CLI usage.
zechenzhangAGI
Skill Author
Loading SKILL.md…
