Overview

Site-wide statistics across every approved benchmark run.

415 last 30d

1,504Runs

58 last 30d

144Contributors

296Models benchmarked

71Hardware setups

51tok/sMedian throughput

2,665tok/sPeak throughput

Activity

Benchmark activity

Approved runs per month

1,504last 12 months

JulAugSepOctNovDecJanFebMarAprMayJun

Performance

Fastest hardware

H200 SXM · 496 tok/s

H200 SXM496 tok/s

H200 NVL333 tok/s

RTX A5000261 tok/s

RTX 3060 Ti225 tok/s

RTX 5070 Ti221 tok/s

RTX A6000131 tok/s

RTX 3090 Ti119 tok/s

Strix Halo Radeon 8060S105 tok/s

RTX 5090104 tok/s

RTX 5060 Ti101 tok/s

Throughput by engine

hipfire · 155 tok/s

hipfire155 tok/s

SGLang88 tok/s

lmstudio85 tok/s

mlx83 tok/s

ollama69 tok/s

vllm65 tok/s

llama.cpp46 tok/s

Popularity

Most benchmarked hardware

Intel Arc Pro B70 · 238

Intel Arc Pro B70238

Ryzen AI Max 395216

Radeon AI Pro R9700163

RTX 3090141

GB10 Grace Blackwell72

RTX 306072

RX 7900 XTX58

RTX 509054

RTX PRO 6000 Blackwell41

RTX 308037

M5 Max29

GTX 1080 Ti27

Most benchmarked models

Qwen3.6-35B-A3B · 92

Qwen3.6-35B-A3B92

Qwen3.6-27B83

MiniMax-M2.7-int4-AutoRound60

gemma-4-26B-A4B-it-GGUF54

Qwen3.6-27B45

MiniMax-M2.740

Qwen3.6-27B-MTP-GGUF34

Ornstein3.6-27B-MTP-NSC-ACE-SABER-GGUF33

Qwen3.6-27B-GGUF31

Qwen3.5-9B27

Qwen3.6-35B-A3B-MTP-GGUF23

gemma-4-31B-it22

Distribution

Most popular quantizations

Q4_K_M · 421

Q4_K_M421

Q8_0115

Q4_072

NVFP468

INT4 AutoRound W4A1665

Q4_K_XL54

UD-Q4_K_XL48

fp843

Q5_K_M40

MQ436

Most popular model sizes

15–35B · 699

<1B4

1–4B60

4–8B92

8–15B204

15–35B699

35–70B250

70–140B90

140B+76

Unknown29

Most popular inference engines

llama.cpp · 1,009

llama.cpp1,009

vllm298

ollama89

hipfire41

mlx39

SGLang15

lmstudio13

Context lengths

≤2K · 595

≤2K595

4K145

8K157

16K40

32K143

64K65

128K+359

Hardware classes

Discrete GPU · 1,055

Discrete GPU1,055

Unified memory433

CPU only16

Architecture mix

Dense · 1,170

Dense1,170

Mixture-of-Experts334

Top contributors

steveseguin · 213

steveseguin213

1337Hero143

gdwallasign94

crown90

AngkorAI67

Skiipy45

schuttdev39

basecampbernie34

gkraker0432

snnn29