Matrix runs
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.
All runs 1,102
Newest first. Page 37 of 45.- Peak decode
- 3,178.6 tok/s
- Peak prefill
- 42.6K tok/s
- Peak decode
- 4,826.6 tok/s
- Peak prefill
- 33.6K tok/s
- Peak decode
- 5,199.9 tok/s
- Peak prefill
- 35.2K tok/s
- Peak decode
- 5,228.0 tok/s
- Peak prefill
- 42.4K tok/s
- Peak decode
- 1,466.6 tok/s
- Peak prefill
- 44K tok/s
- Peak decode
- 1,618.0 tok/s
- Peak prefill
- 33.2K tok/s
- Peak decode
- 1,695.8 tok/s
- Peak prefill
- 34.6K tok/s
- Peak decode
- 1,966.7 tok/s
- Peak prefill
- 42.6K tok/s
- Peak decode
- 3,800.8 tok/s
- Peak prefill
- 23.7K tok/s
- Peak decode
- 4,021.1 tok/s
- Peak prefill
- 24.2K tok/s
- Peak decode
- 3,595.4 tok/s
- Peak prefill
- 29.9K tok/s
- Peak decode
- 872.0 tok/s
- Peak prefill
- 8,929 tok/s
- gpt-oss-safeguard-20bH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 7,201.8 tok/s
- Peak prefill
- 45.2K tok/s
- gpt-oss-safeguard-120bH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 3,931.6 tok/s
- Peak prefill
- 29.7K tok/s
- gpt-oss-20bH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 6,952.6 tok/s
- Peak prefill
- 31.7K tok/s
- gpt-oss-120bH200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · MXFP45 contexts × 8 concurrency levels
- Peak decode
- 3,648.8 tok/s
- Peak prefill
- 18K tok/s
- Qwen3.6-35B-A3B-NVFP4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 5,941.7 tok/s
- Peak prefill
- 34.4K tok/s
- Qwen3.6-27B-NVFP4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 2,301.0 tok/s
- Peak prefill
- 6,619 tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 5,827.8 tok/s
- Peak prefill
- 46.3K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 7,888.9 tok/s
- Peak prefill
- 43.3K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP4+FP88 contexts × 8 concurrency levels
- Peak decode
- 5,854.1 tok/s
- Peak prefill
- 43.6K tok/s
- NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · BF168 contexts × 8 concurrency levels
- Peak decode
- 5,929.6 tok/s
- Peak prefill
- 44.7K tok/s
- NVIDIA-Nemotron-3-Super-120B-A12B-FP8H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · FP88 contexts × 8 concurrency levels
- Peak decode
- 4,357.4 tok/s
- Peak prefill
- 31.5K tok/s
- NVIDIA-Nemotron-3-Super-120B-A12B-BF16H200 NVL 141GB ×2 · vllm 0.26.1rc1.dev638+g52be12cfa · BF168 contexts × 8 concurrency levels
- Peak decode
- 1,611.9 tok/s
- Peak prefill
- 24.1K tok/s
- Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4H200 NVL 141GB · vllm 0.26.1rc1.dev638+g52be12cfa · NVFP4+FP86 contexts × 8 concurrency levels
- Peak decode
- 4,286.1 tok/s
- Peak prefill
- 35.5K tok/s