Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 797
Newest first. Page 20 of 32.- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · BF166 contexts × 8 concurrency levels
- Peak decode
- 7,931.5 tok/s
- Peak prefill
- 34.8K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,547.2 tok/s
- Peak prefill
- 36.6K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.dev1+g50a061f79 · BF166 contexts × 8 concurrency levels
- Peak decode
- 4,373.4 tok/s
- Peak prefill
- 31K tok/s
- Qwen3.8-Flash-NextH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · BF166 contexts × 8 concurrency levels
- Peak decode
- 6,511.0 tok/s
- Peak prefill
- 36.7K tok/s
- DeepSeek-V4.1-FlashH200 SXM 141GB ×4 · vllm 0.1.1.dev5+ge77daef89 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 4,258.2 tok/s
- Peak prefill
- 16.2K tok/s
- DeepSeek-V4.1-FlashH200 SXM 141GB ×8 · vllm 0.1.1.dev5+ge77daef89 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 5,277.2 tok/s
- Peak prefill
- 20.6K tok/s
- DeepSeek-V4.1-FlashH200 SXM 141GB ×8 · vllm 0.1.1.dev5+ge77daef89 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 3,016.1 tok/s
- Peak prefill
- 20.1K tok/s
- DeepSeek-V4.1-FlashH200 SXM 141GB ×8 · vllm 0.1.1.dev5+ge77daef89 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 2,980.0 tok/s
- Peak prefill
- 20.1K tok/s
- Peak decode
- 1,335.6 tok/s
- Peak prefill
- 29.8K tok/s
- Peak decode
- 3,200.8 tok/s
- Peak prefill
- 18.2K tok/s
- Qwen3.8-Flash-Next-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 5,893.3 tok/s
- Peak prefill
- 28.5K tok/s
- Qwen3.8-27B-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 6,984.1 tok/s
- Peak prefill
- 26.5K tok/s
- Qwen3.5-397B-A17B-NVFP4-V2H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 2,363.0 tok/s
- Peak prefill
- 15K tok/s
- Qwen3.5-397B-A17B-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 2,249.8 tok/s
- Peak prefill
- 14.6K tok/s
- NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4H200 SXM 141GB ×4 · vllm 0.28.1rc1.dev562+g9c2d21046 · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 1,329.6 tok/s
- Peak prefill
- 10.1K tok/s
- Peak decode
- 11.5K tok/s
- Peak prefill
- 63.1K tok/s
- Peak decode
- 11.3K tok/s
- Peak prefill
- 63K tok/s
- MiniMax-M2.7H200 SXM 141GB ×4 · vllm 0.19.1rc1.dev203+g0f3ce4c74 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,914.9 tok/s
- Peak prefill
- 31.9K tok/s
- Peak decode
- 2,726.2 tok/s
- Peak prefill
- 34K tok/s
- Peak decode
- 3,640.4 tok/s
- Peak prefill
- 28.1K tok/s
- Peak decode
- 2,668.4 tok/s
- Peak prefill
- 33.7K tok/s
- MiniMax-M2.1H200 SXM 141GB ×4 · vllm 0.19.1rc1.dev203+g0f3ce4c74 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,829.6 tok/s
- Peak prefill
- 31.5K tok/s
- Peak decode
- 2,698.3 tok/s
- Peak prefill
- 33.8K tok/s
- MiniMax-M2H200 SXM 141GB ×4 · vllm 0.19.1rc1.dev203+g0f3ce4c74 · FP86 contexts × 8 concurrency levels
- Peak decode
- 2,907.7 tok/s
- Peak prefill
- 31.4K tok/s
- Peak decode
- 2,744.7 tok/s
- Peak prefill
- 33.5K tok/s