ModelsLeaderboardHardwareEvalsTrainRentalsAPI Docs
Language

HellaSwag

Official

Sentence completion benchmark testing grounded commonsense inference. Models must pick the most plausible continuation of an activity description.

Source
Category: reasoningEval type: LM-Eval runVersion: v1.0Submitted by: Community
LM-Eval run: run lm-eval-harness locally, then submit the task scores and optional artifacts to POST /api/evals/runs.

Eval Details

Scoring
Exact Match
Aggregation
Mean
Direction
Higher is better
Tasks
1 task

Default Run Config

Seed: 42FewShot: 10
TaskDatasetWeightShotsMax Tokens
HellaSwag
hellaswag
Rowan/hellaswag / validation110-shot—

Leaderboard— best run per model

#ModelScoreQuantHardware
Ornstein3.6-35B-A3B
Qwen
81.0%
Q4_K_MRTX 3090
1 reaction

Task Breakdown— top model

hellaswag
81.0%
10-shot · 0 samples