ModelsLeaderboardHardwareEvalsTrainRentalsAPI Docs
Language

HumanEval 0-shot

OpenAI HumanEval via EleutherAI lm-evaluation-harness task humaneval, 0-shot, pass@k code-generation scoring.

Source
Category: codingEval type: LM-Eval runVersion: v1.0Submitted by: banana_baeee
LM-Eval run: run lm-eval-harness locally, then submit the task scores and optional artifacts to POST /api/evals/runs.

Eval Details

Scoring
Pass At K
Aggregation
Mean
Direction
Higher is better
Tasks
1 task
TaskDatasetWeightShotsMax Tokens
HumanEval
humaneval
Not specified1Default—

Leaderboard— best run per model

No approved results yet. Submit a run via the API.