Hardware
112 hardware groups · 199 total
RTX 3090
Discrete GPU
824 speed tests total51 users
Best710 tok/s
Median57.5 tok/s
Min5.5 tok/s
View speed tests →
10x variant1 run
Best22.1 tok/s
Median22.1 tok/s
Min22.1 tok/s
8x variant24 runs
Best404 tok/s
Median93.3 tok/s
Min5.5 tok/s
7x variant2 runs
Best50.8 tok/s
Median42.3 tok/s
Min33.9 tok/s
6x variant1 run
Best47.4 tok/s
Median47.4 tok/s
Min47.4 tok/s
4x variant13 runs
Best152 tok/s
Median51.3 tok/s
Min35.3 tok/s
3x variant1 run
Best231 tok/s
Median231 tok/s
Min231 tok/s
2x variant118 runs
Best314 tok/s
Median70.9 tok/s
Min14.4 tok/s
Single GPU (1x)664 runs
Best710 tok/s
Median56.2 tok/s
Min5.7 tok/s
CMP 170HX
Discrete GPU
1,932 speed tests total5 users
Best3.3k tok/s
Median68.3 tok/s
Min11.4 tok/s
View speed tests →
10x variant29 runs
Best29.7 tok/s
Median29.7 tok/s
Min29.7 tok/s
8x variant37 runs
Best30.6 tok/s
Median30.6 tok/s
Min30.6 tok/s
6x variant10 runs
Best169 tok/s
Median113 tok/s
Min104 tok/s
4x variant17 runs
Best193 tok/s
Median90.6 tok/s
Min78.5 tok/s
2x variant20 runs
Best138 tok/s
Median64.6 tok/s
Min26.1 tok/s
Single GPU (1x)1,819 runs
Best3.3k tok/s
Median71.1 tok/s
Min11.4 tok/s
T4
Discrete GPU
4 speed tests total1 user
Best59.3 tok/s
Median54.0 tok/s
Min46.6 tok/s
View speed tests →