Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 1,102
Newest first. Page 10 of 45.- Peak decode
- 574.1 tok/s
- Peak prefill
- 3,265 tok/s
- Phi-4-reasoning-plus-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
- Peak decode
- 1,423.6 tok/s
- Peak prefill
- 3,210 tok/s
- granite-4.2-30b-fp8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP85 contexts × 8 concurrency levels
- Peak decode
- 282.4 tok/s
- Peak prefill
- 2,003 tok/s
- Peak decode
- 580.6 tok/s
- Peak prefill
- 5,731 tok/s
- NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit5 contexts × 8 concurrency levels
- Peak decode
- 1,310.2 tok/s
- Peak prefill
- 4,010 tok/s
- Reka-Flash-3-21B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic3 contexts × 8 concurrency levels
- Peak decode
- 1,012.0 tok/s
- Peak prefill
- 2,732 tok/s
- Peak decode
- 554.0 tok/s
- Peak prefill
- 5,971 tok/s
- Qwen3.6-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 747.0 tok/s
- Peak prefill
- 1,693 tok/s
- gemma-4-26B-A4B-it-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
- Peak decode
- 2,919.3 tok/s
- Peak prefill
- 7,265 tok/s
- Peak decode
- 2,813.1 tok/s
- Peak prefill
- 5,961 tok/s
- NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
- Peak decode
- 1,741.8 tok/s
- Peak prefill
- 6,443 tok/s
- Qwen3.5-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 769.8 tok/s
- Peak prefill
- 1,664 tok/s
- Qwen3.6-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 823.9 tok/s
- Peak prefill
- 1,681 tok/s
- gemma-3-12b-it-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
- Peak decode
- 372.4 tok/s
- Peak prefill
- 4,968 tok/s
- NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3090 24GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
- Peak decode
- 1,689.9 tok/s
- Peak prefill
- 4,990 tok/s
- Qwen3-Coder-30B-A3B-Instruct-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,564.2 tok/s
- Peak prefill
- 8,096 tok/s
- Qwen3.5-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 796.6 tok/s
- Peak prefill
- 1,725 tok/s
- Peak decode
- 1,164.5 tok/s
- Peak prefill
- 6,836 tok/s
- Peak decode
- 692.1 tok/s
- Peak prefill
- 5,033 tok/s
- Qwen3.6-35B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,234.0 tok/s
- Peak prefill
- 6,676 tok/s
- Peak decode
- 737.3 tok/s
- Peak prefill
- 5,772 tok/s
- Peak decode
- 961.2 tok/s
- Peak prefill
- 7,105 tok/s
- Tess-4.5-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 672.7 tok/s
- Peak prefill
- 2,107 tok/s
- Fara1.5-4B-AutoRound-W4A16-RTNRTX 3090 24GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
- Peak decode
- 3,160.9 tok/s
- Peak prefill
- 10.6K tok/s
- Qwen3.6-35B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,936.7 tok/s
- Peak prefill
- 6,578 tok/s