Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
Matching runs 699
Submitted by @keennay. Newest first. Page 19 of 28.- Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,581.0 tok/s
- Peak prefill
- 41.3K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,855.1 tok/s
- Peak prefill
- 10.3K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,067.6 tok/s
- Peak prefill
- 10.4K tok/s
- Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,315.5 tok/s
- Peak prefill
- 14.3K tok/s
- Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,574.4 tok/s
- Peak prefill
- 14.6K tok/s
- Peak decode
- 7,520.1 tok/s
- Peak prefill
- 30.4K tok/s
- Peak decode
- 6,571.6 tok/s
- Peak prefill
- 31.7K tok/s
- Qwen3.5-9B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,561.1 tok/s
- Peak prefill
- 29.7K tok/s
- Qwen3.5-9B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,570.2 tok/s
- Peak prefill
- 32K tok/s
- Peak decode
- 8,578.7 tok/s
- Peak prefill
- 45.5K tok/s
- Peak decode
- 7,436.0 tok/s
- Peak prefill
- 48.3K tok/s
- Qwen3.5-4B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,196.3 tok/s
- Peak prefill
- 46.6K tok/s
- Qwen3.5-4B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,441.6 tok/s
- Peak prefill
- 46.8K tok/s
- Qwen3.5-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,082.0 tok/s
- Peak prefill
- 41.1K tok/s
- Qwen3.5-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,284.1 tok/s
- Peak prefill
- 39.3K tok/s
- Qwen3.5-35B-A3B-GPTQ-Int4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,829.5 tok/s
- Peak prefill
- 7,807 tok/s
- Qwen3.5-35B-A3B-GPTQ-Int4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,037.9 tok/s
- Peak prefill
- 8,035 tok/s
- Qwen3.5-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 7,955.5 tok/s
- Peak prefill
- 44K tok/s
- Qwen3.5-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,535.7 tok/s
- Peak prefill
- 43.5K tok/s
- Qwen3.5-35B-A3B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,905.2 tok/s
- Peak prefill
- 41.3K tok/s
- Qwen3.5-35B-A3B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,919.3 tok/s
- Peak prefill
- 39.7K tok/s
- Peak decode
- 15.2K tok/s
- Peak prefill
- 90.1K tok/s
- Peak decode
- 15.5K tok/s
- Peak prefill
- 91.7K tok/s
- Qwen3.5-2B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 11.5K tok/s
- Peak prefill
- 87K tok/s
- Qwen3.5-2B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 15.4K tok/s
- Peak prefill
- 96K tok/s