Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 833
Newest first. Page 30 of 34.- Trinity-Mini-BaseH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF165 contexts × 8 concurrency levels
- Peak decode
- 5,521.0 tok/s
- Peak prefill
- 44.1K tok/s
- Trinity-Large-Thinking-W4A16H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · INT46 contexts × 8 concurrency levels
- Peak decode
- 1,265.3 tok/s
- Peak prefill
- 16.3K tok/s
- Olmo-3.1-7B-RL-Zero-MathH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,419.5 tok/s
- Peak prefill
- 31.5K tok/s
- Olmo-3.1-7B-RL-Zero-CodeH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,423.2 tok/s
- Peak prefill
- 30.2K tok/s
- Olmo-3.1-32B-ThinkH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,639.9 tok/s
- Peak prefill
- 8,490 tok/s
- Olmo-3.1-32B-InstructH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,638.4 tok/s
- Peak prefill
- 8,567 tok/s
- Olmo-3.1-32B-Instruct-SFTH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,641.7 tok/s
- Peak prefill
- 8,414 tok/s
- Olmo-3.1-32B-Instruct-DPOH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,640.0 tok/s
- Peak prefill
- 8,762 tok/s
- Olmo-3-7B-ThinkH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,418.8 tok/s
- Peak prefill
- 32K tok/s
- Olmo-3-7B-InstructH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,420.0 tok/s
- Peak prefill
- 31K tok/s
- Olmo-3-32B-ThinkH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF164 contexts × 8 concurrency levels
- Peak decode
- 1,639.1 tok/s
- Peak prefill
- 8,497 tok/s
- Peak decode
- 2,383.6 tok/s
- Peak prefill
- 35K tok/s
- ZAYA1-8B-legacyH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF165 contexts × 8 concurrency levels
- Peak decode
- 75.6 tok/s
- Peak prefill
- 45.6K tok/s
- Qwen3.8-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 654.5 tok/s
- Peak prefill
- 9,751 tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,412.6 tok/s
- Peak prefill
- 32.4K tok/s
- Qwen3.6-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 698.2 tok/s
- Peak prefill
- 9,796 tok/s
- Qwen3.5-9BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,712.8 tok/s
- Peak prefill
- 26.4K tok/s
- Qwen3.5-4BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,657.1 tok/s
- Peak prefill
- 38.2K tok/s
- Qwen3.5-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,945.8 tok/s
- Peak prefill
- 32.4K tok/s
- Qwen3.5-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,550.5 tok/s
- Peak prefill
- 9,165 tok/s
- gemma-4-12B-itH200 NVL 141GB · sglang 0.0.0.dev16562+g6283bde44 · BF166 contexts × 8 concurrency levels
- Peak decode
- 611.1 tok/s
- Peak prefill
- 9,653 tok/s
- gemma-4-26B-A4B-itH200 NVL 141GB · sglang 0.5.6.post3.dev9269+g834400705 · BF166 contexts × 8 concurrency levels
- Peak decode
- 497.4 tok/s
- Peak prefill
- 10.9K tok/s
- Step-3.7-Flash-NVFP4H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9269+g834400705 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 1,432.7 tok/s
- Peak prefill
- 17.3K tok/s
- Step-3.7-Flash-FP8H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9269+g834400705 · FP86 contexts × 8 concurrency levels
- Peak decode
- 889.0 tok/s
- Peak prefill
- 24.8K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9265+g05c584c44 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,622.2 tok/s
- Peak prefill
- 37.2K tok/s