로컬 LLM 품질을 평가하기 위한 커뮤니티 벤치마크 스위트입니다. API를 통해 결과를 제출하세요.
Mostly Basic Python Problems — 500 crowd-sourced Python programming problems with automated test cases. Broader coverage than HumanEval.
OpenAI's Python function completion benchmark. 164 hand-crafted problems with unit tests measuring pass@1 code synthesis accuracy.