Hardware
121 hardware groups · 226 total
RTX 3090
Discrete GPU
1,517 speed tests total55 users
Best5.0k tok/s
Median81.9 tok/s
Min5.5 tok/s
View speed tests →
10x variant1 run
Best22.1 tok/s
Median22.1 tok/s
Min22.1 tok/s
8x variant26 runs
Best404 tok/s
Median86.2 tok/s
Min5.5 tok/s
7x variant2 runs
Best50.8 tok/s
Median42.3 tok/s
Min33.9 tok/s
6x variant1 run
Best47.4 tok/s
Median47.4 tok/s
Min47.4 tok/s
4x variant13 runs
Best152 tok/s
Median51.3 tok/s
Min35.3 tok/s
3x variant1 run
Best231 tok/s
Median231 tok/s
Min231 tok/s
2x variant219 runs
Best5.0k tok/s
Median143 tok/s
Min14.4 tok/s
Single GPU (1x)1,254 runs
Best710 tok/s
Median77.3 tok/s
Min5.7 tok/s
CMP 170HX
Discrete GPU
2,638 speed tests total5 users
Best3.3k tok/s
Median76.8 tok/s
Min8.4 tok/s
View speed tests →
10x variant29 runs
Best29.7 tok/s
Median29.7 tok/s
Min29.7 tok/s
8x variant122 runs
Best532 tok/s
Median57.2 tok/s
Min22.1 tok/s
6x variant10 runs
Best169 tok/s
Median113 tok/s
Min104 tok/s
4x variant90 runs
Best1.2k tok/s
Median163 tok/s
Min71.0 tok/s
2x variant102 runs
Best151 tok/s
Median66.6 tok/s
Min8.4 tok/s
Single GPU (1x)2,285 runs
Best3.3k tok/s
Median77.1 tok/s
Min8.7 tok/s
T4
Discrete GPU
7 speed tests total1 user
Best59.3 tok/s
Median46.6 tok/s
Min30.5 tok/s
View speed tests →
