Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 804
Newest first. Page 23 of 33.- Qwen3.8-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,782.2 tok/s
- Peak prefill
- 14.1K tok/s
- Qwen3.8-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,570.2 tok/s
- Peak prefill
- 14.5K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,835.3 tok/s
- Peak prefill
- 40.5K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,351.1 tok/s
- Peak prefill
- 39K tok/s
- Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 7,688.1 tok/s
- Peak prefill
- 43.1K tok/s
- Qwen3.6-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,581.0 tok/s
- Peak prefill
- 41.3K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,855.1 tok/s
- Peak prefill
- 10.3K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,067.6 tok/s
- Peak prefill
- 10.4K tok/s
- Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,315.5 tok/s
- Peak prefill
- 14.3K tok/s
- Qwen3.6-27B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,574.4 tok/s
- Peak prefill
- 14.6K tok/s
- Peak decode
- 7,520.1 tok/s
- Peak prefill
- 30.4K tok/s
- Peak decode
- 6,571.6 tok/s
- Peak prefill
- 31.7K tok/s
- Qwen3.5-9B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,561.1 tok/s
- Peak prefill
- 29.7K tok/s
- Qwen3.5-9B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,570.2 tok/s
- Peak prefill
- 32K tok/s
- Peak decode
- 8,578.7 tok/s
- Peak prefill
- 45.5K tok/s
- Peak decode
- 7,436.0 tok/s
- Peak prefill
- 48.3K tok/s
- Qwen3.5-4B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,196.3 tok/s
- Peak prefill
- 46.6K tok/s
- Qwen3.5-4B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,441.6 tok/s
- Peak prefill
- 46.8K tok/s
- Qwen3.5-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,082.0 tok/s
- Peak prefill
- 41.1K tok/s
- Qwen3.5-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,284.1 tok/s
- Peak prefill
- 39.3K tok/s
- Qwen3.5-35B-A3B-GPTQ-Int4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,829.5 tok/s
- Peak prefill
- 7,807 tok/s
- Qwen3.5-35B-A3B-GPTQ-Int4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,037.9 tok/s
- Peak prefill
- 8,035 tok/s
- Qwen3.5-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 7,955.5 tok/s
- Peak prefill
- 44K tok/s
- Qwen3.5-35B-A3B-FP8H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · FP86 contexts × 8 concurrency levels
- Peak decode
- 6,535.7 tok/s
- Peak prefill
- 43.5K tok/s
- Qwen3.5-35B-A3B-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,905.2 tok/s
- Peak prefill
- 41.3K tok/s