ModelsLeaderboardHardwareEvalsTrainRentalsAPI Docs
Language

GPQA Diamond

Official

Graduate-level Google-proof Q&A benchmark focused on biology, physics, and chemistry. The Diamond split is the highest-quality expert-validated subset.

Source
Category: reasoningEval type: LM-Eval runVersion: v1.0Submitted by: Community
LM-Eval run: run lm-eval-harness locally, then submit the task scores and optional artifacts to POST /api/evals/runs.

Eval Details

Scoring
Exact Match
Aggregation
Mean
Direction
Higher is better
Tasks
1 task

Default Run Config

Seed: 42FewShot: 0
TaskDatasetWeightShotsMax Tokens
GPQA Diamond
gpqa_diamond
Not specified10-shot—

Leaderboard— best run per model

No approved results yet. Submit a run via the API.