Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 826
Newest first. Page 28 of 34.- LFM2.5-230MH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF165 contexts × 8 concurrency levels
- Peak decode
- 13.3K tok/s
- Peak prefill
- 192.2K tok/s
- LFM2.5-2.6BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF165 contexts × 8 concurrency levels
- Peak decode
- 12.7K tok/s
- Peak prefill
- 66.2K tok/s
- LFM2.5-1.2B-InstructH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF165 contexts × 8 concurrency levels
- Peak decode
- 22.2K tok/s
- Peak prefill
- 108.8K tok/s
- Qwen3.6-35B-A3B-int4-mixed-AutoRoundH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT4+BF166 contexts × 8 concurrency levels
- Peak decode
- 7,320.8 tok/s
- Peak prefill
- 35.1K tok/s
- Qwen3.6-35B-A3B-int4-mixed-AutoRoundH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT4+BF166 contexts × 8 concurrency levels
- Peak decode
- 5,966.8 tok/s
- Peak prefill
- 35.1K tok/s
- Qwen3.6-27B-int4-AutoRoundH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT4+BF166 contexts × 8 concurrency levels
- Peak decode
- 2,879.6 tok/s
- Peak prefill
- 9,975 tok/s
- Qwen3.6-27B-int4-AutoRoundH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT4+BF166 contexts × 8 concurrency levels
- Peak decode
- 2,904.9 tok/s
- Peak prefill
- 9,928 tok/s
- gemma-4-31B-it-int4-AutoRound-V2H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT4+BF166 contexts × 8 concurrency levels
- Peak decode
- 1,219.6 tok/s
- Peak prefill
- 7,966 tok/s
- gemma-4-12B-it-int8-AutoRoundH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT85 contexts × 8 concurrency levels
- Peak decode
- 4,085.8 tok/s
- Peak prefill
- 17.9K tok/s
- gemma-4-12B-it-int4-AutoRoundH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 4,246.4 tok/s
- Peak prefill
- 18.7K tok/s
- Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,499.2 tok/s
- Peak prefill
- 10K tok/s
- Ling-3.0-tinyH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF165 contexts × 8 concurrency levels
- Peak decode
- 13K tok/s
- Peak prefill
- 69.5K tok/s
- Ling-3.0-tiny-int4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · INT45 contexts × 8 concurrency levels
- Peak decode
- 15K tok/s
- Peak prefill
- 60.9K tok/s
- Ling-3.0-tiny-baseH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 12.6K tok/s
- Peak prefill
- 62.3K tok/s
- Ling-3.0-tiny-base-midtrainH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 12.6K tok/s
- Peak prefill
- 67.7K tok/s
- Ling-3.0-flash-int4H200 NVL 141GB ×2 · vllm 0.27.2rc1.dev113+g5cecfc013 · INT46 contexts × 8 concurrency levels
- Peak decode
- 6,337.6 tok/s
- Peak prefill
- 28.3K tok/s
- Ling-3.0-flash-fp8H200 NVL 141GB ×2 · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 5,243.0 tok/s
- Peak prefill
- 33.3K tok/s
- Ling-3.0-flash-fp4H200 NVL 141GB ×2 · vllm 0.27.2rc1.dev113+g5cecfc013 · MXFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 5,912.7 tok/s
- Peak prefill
- 28.2K tok/s
- Ling-2.6-flashH200 NVL 141GB ×2 · vllm 0.27.2rc1.dev113+g5cecfc013 · BF165 contexts × 8 concurrency levels
- Peak decode
- 4,663.2 tok/s
- Peak prefill
- 31.7K tok/s
- Ling-2.6-flash-int4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · INT45 contexts × 8 concurrency levels
- Peak decode
- 4,238.6 tok/s
- Peak prefill
- 16.4K tok/s
- Ling-2.6-flash-fp8H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP85 contexts × 8 concurrency levels
- Peak decode
- 3,739.3 tok/s
- Peak prefill
- 22.7K tok/s
- Ling-2.6-flash-baseH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,665.7 tok/s
- Peak prefill
- 22.4K tok/s
- Peak decode
- 2,135.2 tok/s
- Peak prefill
- 27.3K tok/s
- Peak decode
- 4,193.1 tok/s
- Peak prefill
- 49.7K tok/s
- Peak decode
- 672.0 tok/s
- Peak prefill
- 9,269 tok/s