GSM8K

Official

Grade School Math 8K — 8,500 grade-school math word problems requiring multi-step arithmetic reasoning. Standard benchmark for math reasoning capability.

Source

Category: mathEval type: LM-Eval runVersion: v1.0Submitted by: Community

LM-Eval run: run lm-eval-harness locally, then submit the task scores and optional artifacts to POST /api/evals/runs.

Eval Details

Scoring

Exact Match

Aggregation

Mean

Direction

Higher is better

Tasks

1 task

Default Run Config

Seed: 42FewShot: 5

Task	Dataset	Weight	Shots	Max Tokens
GSM8K gsm8k	gsm8k / main / test	1	5-shot	—

Leaderboard— best run per model

#	Model	Score	Quant	Hardware
	Ornstein3.6-35B-A3B Qwen	95.0%	Q4_K_M	RTX 3090

Task Breakdown— top model

gsm8k

95.0%

5-shot · 0 samples