Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 1,102
Newest first. Page 8 of 45.- Ornith-1.5-35B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,073.6 tok/s
- Peak prefill
- 7,202 tok/s
- Sage-A1.5-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,232.1 tok/s
- Peak prefill
- 7,518 tok/s
- Ornith-1.5-35B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,797.6 tok/s
- Peak prefill
- 7,797 tok/s
- Nanbeige4.3-33B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP85 contexts × 8 concurrency levels
- Peak decode
- 261.6 tok/s
- Peak prefill
- 1,851 tok/s
- NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3090 24GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
- Peak decode
- 1,679.8 tok/s
- Peak prefill
- 4,527 tok/s
- Tess-4.5-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 724.5 tok/s
- Peak prefill
- 2,062 tok/s
- Fara1.5-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 680.3 tok/s
- Peak prefill
- 2,125 tok/s
- internlm_internlm3-8b-instruct-autoround-int8-gs128-symRTX 3090 24GB · vllm 0.32.0 · AutoRound INT8 W8A16 gs1283 contexts × 8 concurrency levels
- Peak decode
- 1,127.1 tok/s
- Peak prefill
- 4,961 tok/s
- Peak decode
- 1,557.4 tok/s
- Peak prefill
- 2,935 tok/s
- Peak decode
- 258.4 tok/s
- Peak prefill
- 6,919 tok/s
- Qwen-AgentWorld-35B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,849.5 tok/s
- Peak prefill
- 7,749 tok/s
- Qwen3-1.7B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
- Peak decode
- 3,788.8 tok/s
- Peak prefill
- 18K tok/s
- Peak decode
- 378.8 tok/s
- Peak prefill
- 7,169 tok/s
- Peak decode
- 1,663.1 tok/s
- Peak prefill
- 3,384 tok/s
- Llama-3.2-3B-Instruct-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
- Peak decode
- 2,507.0 tok/s
- Peak prefill
- 13.6K tok/s
- DeepSeek-R1-Distill-Qwen-32B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
- Peak decode
- 244.1 tok/s
- Peak prefill
- 1,856 tok/s
- Fara1.5-4B-AutoRound-W4A16-RTNRTX 3090 24GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
- Peak decode
- 3,794.2 tok/s
- Peak prefill
- 8,459 tok/s
- Peak decode
- 1,792.5 tok/s
- Peak prefill
- 37.9K tok/s
- Fara1.5-4B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,940.8 tok/s
- Peak prefill
- 10.3K tok/s
- Seed-OSS-36B-Instruct-FP8-StaticCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 static5 contexts × 8 concurrency levels
- Peak decode
- 224.8 tok/s
- Peak prefill
- 1,545 tok/s
- Peak decode
- 684.9 tok/s
- Peak prefill
- 21.8K tok/s
- Peak decode
- 1,601.6 tok/s
- Peak prefill
- 12.3K tok/s
- Agents-A1-4B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,812.8 tok/s
- Peak prefill
- 10.8K tok/s
- Qwen3-0.6B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
- Peak decode
- 3,926.4 tok/s
- Peak prefill
- 27.5K tok/s
- Peak decode
- 3,858.9 tok/s
- Peak prefill
- 8,430 tok/s