Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 1,102
Newest first. Page 36 of 45.- Peak decode
- 15.5K tok/s
- Peak prefill
- 91.7K tok/s
- Qwen3.5-2B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 11.5K tok/s
- Peak prefill
- 87K tok/s
- Qwen3.5-2B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 15.4K tok/s
- Peak prefill
- 96K tok/s
- Qwen3.5-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,103.9 tok/s
- Peak prefill
- 10.3K tok/s
- Qwen3.5-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,071.0 tok/s
- Peak prefill
- 10.4K tok/s
- Qwen3.5-27B-GPTQ-Int4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 3,104.6 tok/s
- Peak prefill
- 10.1K tok/s
- Qwen3.5-27B-GPTQ-Int4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 2,283.8 tok/s
- Peak prefill
- 10K tok/s
- Qwen3.5-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,576.0 tok/s
- Peak prefill
- 14.3K tok/s
- Qwen3.5-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,576.1 tok/s
- Peak prefill
- 14.6K tok/s
- Qwen3.5-122B-A10B-GPTQ-Int4H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 6,342.6 tok/s
- Peak prefill
- 22.8K tok/s
- Qwen3.5-122B-A10B-GPTQ-Int4H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 4,948.7 tok/s
- Peak prefill
- 22.6K tok/s
- Qwen3.5-122B-A10B-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,502.8 tok/s
- Peak prefill
- 29.3K tok/s
- Qwen3.5-122B-A10B-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 5,019.0 tok/s
- Peak prefill
- 29.9K tok/s
- Qwen3.5-0.8BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 15.6K tok/s
- Peak prefill
- 113.3K tok/s
- Qwen3.5-0.8BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 17.1K tok/s
- Peak prefill
- 132.9K tok/s
- Qwen3.5-0.8B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 11.9K tok/s
- Peak prefill
- 118.1K tok/s
- Qwen3.5-0.8B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 17.1K tok/s
- Peak prefill
- 122.8K tok/s
- INTELLECT-3H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · BF165 contexts × 8 concurrency levels
- Peak decode
- 514.0 tok/s
- Peak prefill
- 21.3K tok/s
- INTELLECT-3.1H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · BF165 contexts × 8 concurrency levels
- Peak decode
- 516.6 tok/s
- Peak prefill
- 20.8K tok/s
- INTELLECT-3-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP85 contexts × 8 concurrency levels
- Peak decode
- 1,491.4 tok/s
- Peak prefill
- 24.2K tok/s
- INTELLECT-2H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 625.2 tok/s
- Peak prefill
- 8,458 tok/s
- Peak decode
- 3,232.3 tok/s
- Peak prefill
- 43.2K tok/s
- Peak decode
- 4,818.2 tok/s
- Peak prefill
- 32.2K tok/s
- Peak decode
- 5,181.7 tok/s
- Peak prefill
- 36.2K tok/s
- Peak decode
- 5,258.4 tok/s
- Peak prefill
- 41.2K tok/s