Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 800
Newest first. Page 21 of 32.- Peak decode
- 2,698.3 tok/s
- Peak prefill
- 33.8K tok/s
- MiniMax-M2H200 SXM 141GB ×4 · vllm 0.19.1rc1.dev203+g0f3ce4c74 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,907.7 tok/s
- Peak prefill
- 31.4K tok/s
- Peak decode
- 2,744.7 tok/s
- Peak prefill
- 33.5K tok/s
- DeepSeek-V4-Flash-Vision-ExpH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev317+g1356635d8 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 6,553.4 tok/s
- Peak prefill
- 21.6K tok/s
- DeepSeek-V4-Flash-Vision-ExpH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev317+g1356635d8 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 6,158.3 tok/s
- Peak prefill
- 21.8K tok/s
- DeepSeek-V4-Flash-Vision-ExpH200 SXM 141GB ×4 · sglang 0.5.6.post3.dev9812+gd5a3b7725 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 136.9 tok/s
- Peak prefill
- 21.2K tok/s
- Peak decode
- 2,793.5 tok/s
- Peak prefill
- 22.8K tok/s
- Peak decode
- 8,783.5 tok/s
- Peak prefill
- 41.6K tok/s
- Ling-3.0-flashH200 SXM 141GB ×4 · sglang 0.5.6.post3.dev8719+ge1a24a189 · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,881.5 tok/s
- Peak prefill
- 53.7K tok/s
- command-a-plus-05-2026-bf16H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev455+g22f931fe2 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,354.6 tok/s
- Peak prefill
- 24.6K tok/s
- Trinity-Large-Thinking-FP8-BlockH200 SXM 141GB ×4 · vllm 0.28.1rc1.dev455+g22f931fe2 · FP87 contexts × 8 concurrency levels
- Peak decode
- 3,864.0 tok/s
- Peak prefill
- 33.3K tok/s
- gemma-4-12B-itH200 NVL 141GB · sglang 0.0.0.dev16952+gd5bd07850 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,015.5 tok/s
- Peak prefill
- 10.1K tok/s
- Peak decode
- 4,637.7 tok/s
- Peak prefill
- 60.9K tok/s
- ZAYA1-8B-legacyH200 NVL 141GB · vllm 0.1.dev17064+g8df704c52 · BF165 contexts × 8 concurrency levels
- Peak decode
- 3,057.5 tok/s
- Peak prefill
- 55K tok/s
- ZAYA1-74B-preview-legacyH200 NVL 141GB · vllm 0.1.dev17064+g8df704c52 · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,582.1 tok/s
- Peak prefill
- 20.4K tok/s
- Qwen3.8-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 1,521.1 tok/s
- Peak prefill
- 10.1K tok/s
- Qwen3.6-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,954.8 tok/s
- Peak prefill
- 41.8K tok/s
- Qwen3.6-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 887.0 tok/s
- Peak prefill
- 10.2K tok/s
- Peak decode
- 5,252.1 tok/s
- Peak prefill
- 30.5K tok/s
- Peak decode
- 6,429.2 tok/s
- Peak prefill
- 46.7K tok/s
- Qwen3.5-35B-A3BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 3,465.9 tok/s
- Peak prefill
- 42.5K tok/s
- Qwen3.5-27BH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 2,214.8 tok/s
- Peak prefill
- 10.2K tok/s
- gpt-oss-20bH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 5,651.7 tok/s
- Peak prefill
- 31.1K tok/s
- gpt-oss-120bH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 2,383.3 tok/s
- Peak prefill
- 19K tok/s
- gemma-4-12B-itH200 NVL 141GB · vllm 0.26.1rc1.dev914+gc3dabcddc · BF166 contexts × 8 concurrency levels
- Peak decode
- 5,195.8 tok/s
- Peak prefill
- 19.5K tok/s