Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 1,102
Newest first. Page 44 of 45.- Qwen3.5-122B-A10B-FP8H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,794.2 tok/s
- Peak prefill
- 28.4K tok/s
- Qwen3.5-0.8BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
- Peak decode
- 12.3K tok/s
- Peak prefill
- 99K tok/s
- Qwen3.5-0.8BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
- Peak decode
- 16K tok/s
- Peak prefill
- 98.7K tok/s
- Qwen3.5-0.8B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,199.7 tok/s
- Peak prefill
- 98.2K tok/s
- Qwen3.5-0.8B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
- Peak decode
- 15.7K tok/s
- Peak prefill
- 104.3K tok/s
- INTELLECT-3H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF165 contexts × 8 concurrency levels
- Peak decode
- 607.4 tok/s
- Peak prefill
- 16.3K tok/s
- INTELLECT-3.1H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · BF165 contexts × 8 concurrency levels
- Peak decode
- 604.2 tok/s
- Peak prefill
- 16.4K tok/s
- INTELLECT-2H200 NVL 141GB · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP164 contexts × 8 concurrency levels
- Peak decode
- 481.2 tok/s
- Peak prefill
- 7,964 tok/s
- Laguna-XS.2H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,861.9 tok/s
- Peak prefill
- 34.8K tok/s
- Laguna-XS.2-INT4H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · INT4+INT86 contexts × 8 concurrency levels
- Peak decode
- 2,619.0 tok/s
- Peak prefill
- 32.9K tok/s
- Laguna-XS.2-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,419.2 tok/s
- Peak prefill
- 41.6K tok/s
- Laguna-XS-2.1H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,832.7 tok/s
- Peak prefill
- 34.9K tok/s
- Laguna-XS-2.1-INT4H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · INT4+INT86 contexts × 8 concurrency levels
- Peak decode
- 2,633.3 tok/s
- Peak prefill
- 33.2K tok/s
- Laguna-XS-2.1-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,409.8 tok/s
- Peak prefill
- 41.7K tok/s
- DeepSeek-V4-Flash-DSparkH200 NVL 141GB ×2 · sglang 0.5.16 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 4,695.4 tok/s
- Peak prefill
- 15K tok/s
- DeepSeek-V4-Flash-0731H200 NVL 141GB ×2 · sglang 0.5.16 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 4,020.3 tok/s
- Peak prefill
- 15K tok/s
- Peak decode
- 3,575.4 tok/s
- Peak prefill
- 14.8K tok/s
- Peak decode
- 2,811.9 tok/s
- Peak prefill
- 15K tok/s
- DeepSeek-V4-Flash-DSparkH200 NVL 141GB ×2 · vllm 0.25.0 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 4,201.1 tok/s
- Peak prefill
- 11.6K tok/s
- Peak decode
- 3,978.4 tok/s
- Peak prefill
- 11.7K tok/s
- Peak decode
- 2,877.6 tok/s
- Peak prefill
- 15.2K tok/s
- Laguna-XS-2.1H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,728.0 tok/s
- Peak prefill
- 34.2K tok/s
- Laguna-XS-2.1-INT4H200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · INT4+INT86 contexts × 8 concurrency levels
- Peak decode
- 3,250.7 tok/s
- Peak prefill
- 33.5K tok/s
- Qwen3.8-Flash-Next-FP8H200 NVL 141GB ×2 · vllm 0.1.1.dev5+ge77daef89 · FP86 contexts × 8 concurrency levels
- Peak decode
- 3,324.4 tok/s
- Peak prefill
- 20.7K tok/s
- Qwen3.8-Flash-Next-FP8H200 NVL 141GB ×2 · vllm 0.1.1.dev5+ge77daef89 · FP86 contexts × 8 concurrency levels
- Peak decode
- 4,229.4 tok/s
- Peak prefill
- 21.4K tok/s