Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 801
Newest first. Page 22 of 33.- gemma-4-12B-itH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,195.8 tok/s
- Peak prefill
- 19.5K tok/s
- gemma-4-31B-itH200 NVL 141GB ×2 · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,740.9 tok/s
- Peak prefill
- 14.4K tok/s
- gemma-4-26B-A4B-itH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 10.1K tok/s
- Peak prefill
- 37.3K tok/s
- Step-3.7-Flash-NVFP4H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 3,572.1 tok/s
- Peak prefill
- 18.3K tok/s
- Step-3.7-Flash-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,443.8 tok/s
- Peak prefill
- 24.6K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,302.8 tok/s
- Peak prefill
- 38.6K tok/s
- Qwen3.6-35B-A3B-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 5,833.6 tok/s
- Peak prefill
- 31.1K tok/s
- Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,506.0 tok/s
- Peak prefill
- 41.4K tok/s
- Qwen3.6-35B-A3B-FP8-dynamicH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,065.3 tok/s
- Peak prefill
- 41.4K tok/s
- Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,554.0 tok/s
- Peak prefill
- 14.5K tok/s
- gemma-4-E4B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF165 contexts × 8 concurrency levels
- Peak decode
- 7,744.3 tok/s
- Peak prefill
- 40.6K tok/s
- gemma-4-E2B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF165 contexts × 8 concurrency levels
- Peak decode
- 13.1K tok/s
- Peak prefill
- 62K tok/s
- gemma-4-31B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 616.9 tok/s
- Peak prefill
- 7,892 tok/s
- gemma-4-31B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 1,143.4 tok/s
- Peak prefill
- 4,486 tok/s
- gemma-4-31B-it-FP8-blockH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · FP86 contexts × 8 concurrency levels
- Peak decode
- 1,189.1 tok/s
- Peak prefill
- 11.4K tok/s
- gemma-4-26B-A4B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,389.3 tok/s
- Peak prefill
- 40.2K tok/s
- gemma-4-26B-A4B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 4,355.9 tok/s
- Peak prefill
- 24.7K tok/s
- gemma-4-12B-itH200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,491.4 tok/s
- Peak prefill
- 19.4K tok/s
- gemma-4-12B-it-NVFP4H200 NVL 141GB · vllm 0.27.2rc1.dev113+g5cecfc013 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 3,875.8 tok/s
- Peak prefill
- 11.2K tok/s
- Peak decode
- 115.1 tok/s
- Peak prefill
- 34.5K tok/s
- Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,381.2 tok/s
- Peak prefill
- 10.2K tok/s
- Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,063.9 tok/s
- Peak prefill
- 10.4K tok/s
- Qwen3.8-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,782.2 tok/s
- Peak prefill
- 14.1K tok/s
- Qwen3.8-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,570.2 tok/s
- Peak prefill
- 14.5K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,835.3 tok/s
- Peak prefill
- 40.5K tok/s