ModelsLeaderboardHardwareEvalsTrainRentalsAPI Docs
Language

Probe

Category: knowledgeEval type: LM-Eval runVersion: v1.0Submitted by: banana_baeee
LM-Eval run: run lm-eval-harness locally, then submit the task scores and optional artifacts to POST /api/evals/runs.

Eval Details

Scoring
Exact Match
Aggregation
Weighted Mean
Direction
Higher is better
Tasks
1 task
TaskDatasetWeightShotsMax Tokens
mmlu
mmlu
Not specified1Default

Leaderboard— best run per model

#ModelScoreQuantHardware
Ornstein3.6-35B-A3B
Qwen
81.9%
Q4_K_MRTX 3090

Task Breakdown— top model

mmlu
81.9%
0-shot · 0 samples