Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 1,102
Newest first. Page 15 of 45.- Qwen3.8-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
- Peak decode
- 821.9 tok/s
- Peak prefill
- 1,717 tok/s
- Peak decode
- 171.1 tok/s
- Peak prefill
- 2,179 tok/s
- Peak decode
- 1,813.0 tok/s
- Peak prefill
- 10.5K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP48 contexts × 8 concurrency levels
- Peak decode
- 2,954.9 tok/s
- Peak prefill
- 8,465 tok/s
- DeepSeek-V4.1-Flash-1017RTX PRO 6000 Blackwell Server Edition 96GB ×8 · vllm 0.33.0rc1.dev141 · MXFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 1,772.3 tok/s
- Peak prefill
- 11.1K tok/s
- Tess-4-27B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
- Peak decode
- 1,101.0 tok/s
- Peak prefill
- 1,837 tok/s
- gemma-4-31B-it-qat-w4a16-ctCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · W4A166 contexts × 8 concurrency levels
- Peak decode
- 683.8 tok/s
- Peak prefill
- 1,368 tok/s
- gpt-oss-20bCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 3,816.0 tok/s
- Peak prefill
- 8,773 tok/s
- GLM-5.3-Flash-W4A16-MTPCMP 170HX 64GB ×4 · vllm 0.32.0rc1.dev215+g819146023 · W4A167 contexts × 5 concurrency levels
- Peak decode
- 281.9 tok/s
- Peak prefill
- 6,577 tok/s
- Qwen3.8-27B-INT4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · INT4 W4A16 g1286 contexts × 8 concurrency levels
- Peak decode
- 864.5 tok/s
- Peak prefill
- 1,922 tok/s
- Qwen3.8-27B-INT4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · INT4 W4A16 g1286 contexts × 8 concurrency levels
- Peak decode
- 1,247.4 tok/s
- Peak prefill
- 1,949 tok/s
- granite-4.2-3b-nvfp4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
- Peak decode
- 3,799.2 tok/s
- Peak prefill
- 10.5K tok/s
- Peak decode
- 752.8 tok/s
- Peak prefill
- 10.2K tok/s
- Peak decode
- 1,330.3 tok/s
- Peak prefill
- 11.4K tok/s
- GLM-5.3-Flash-NVFP4H200 NVL 141GB ×2 · vllm 0.1.dev1+g3b5189828 · NVFP48 contexts × 8 concurrency levels
- Peak decode
- 959.4 tok/s
- Peak prefill
- 13.2K tok/s
- GLM-5.3-Flash-NVFP4H200 NVL 141GB ×2 · vllm 0.1.dev1+g3b5189828 · NVFP48 contexts × 8 concurrency levels
- Peak decode
- 1,234.2 tok/s
- Peak prefill
- 13K tok/s
- Qwen3.8-Flash-Next-W4A16Intel Arc Pro B65 32GB ×4 · vllm 0.30.0 · INT4_W4A166 contexts × 3 concurrency levels
- Peak decode
- 242.6 tok/s
- Peak prefill
- 3,961 tok/s
- Qwen3.8-Flash-Next-W4A16Intel Arc Pro B65 32GB ×4 · vllm 0.30.0 · INT4_W4A166 contexts × 3 concurrency levels
- Peak decode
- 158.3 tok/s
- Peak prefill
- 3,856 tok/s
- Qwen3.8-Flash-Next-W4A16Intel Arc Pro B65 32GB ×4 · vllm 0.30.0 · INT4_W4A166 contexts × 3 concurrency levels
- Peak decode
- 158.6 tok/s
- Peak prefill
- 5,306 tok/s
- Qwen3.8-Flash-Next-W4A16Intel Arc Pro B65 32GB ×4 · vllm 0.30.0 · INT4_W4A166 contexts × 3 concurrency levels
- Peak decode
- 246.3 tok/s
- Peak prefill
- 5,206 tok/s
- GLM-5.3-Flash-NVFP4H200 SXM 141GB ×4 · vllm 0.1.dev1+g3b5189828 · NVFP48 contexts × 8 concurrency levels
- Peak decode
- 2,214.4 tok/s
- Peak prefill
- 18.4K tok/s
- Peak decode
- 2,156.0 tok/s
- Peak prefill
- 18.3K tok/s
- NVIDIA-Nemotron-3-Super-120B-A12B-FP8H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9335+gdd15fb57b · FP88 contexts × 8 concurrency levels
- Peak decode
- 2,815.6 tok/s
- Peak prefill
- 15.5K tok/s
- gemma-4-31B-itIntel Arc Pro B65 32GB · vllm 0.31.0+xpu.llmscaler · INT41 context × 5 concurrency levels
- Peak decode
- 207.3 tok/s
- Peak prefill
- —
- Peak decode
- 425.3 tok/s
- Peak prefill
- —