Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 9 of 45.
  • granite-4.2-3b-fp8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP85 contexts × 8 concurrency levels
    Peak decode
    3,789.9 tok/s
    Peak prefill
    11.4K tok/s
    by @Sandberg
  • Llama-3.2-3B-Instruct-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    1,514.0 tok/s
    Peak prefill
    14.6K tok/s
    by @Sandberg
  • Ministral-3-3B-Instruct-2512-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    3,735.6 tok/s
    Peak prefill
    11.5K tok/s
    by @Sandberg
  • NVIDIA-Nemotron-3-Nano-4B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    3,664.8 tok/s
    Peak prefill
    10.4K tok/s
    by @Sandberg
  • NVIDIA-Nemotron-3-Nano-4B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    3,229.6 tok/s
    Peak prefill
    11.6K tok/s
    by @Sandberg
  • gemma-4-26B-A4B-it-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    929.7 tok/s
    Peak prefill
    7,033 tok/s
    by @Sandberg
  • granite-4.2-8b-fp8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP85 contexts × 8 concurrency levels
    Peak decode
    1,636.8 tok/s
    Peak prefill
    6,024 tok/s
    by @Sandberg
  • Qwen3.5-4B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    3,662.7 tok/s
    Peak prefill
    10.6K tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    2,101.8 tok/s
    Peak prefill
    6,307 tok/s
    by @Sandberg
  • Fara1.5-4B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,189.2 tok/s
    Peak prefill
    11.8K tok/s
    by @Sandberg
  • Qwen3.5-4B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    4,738.8 tok/s
    Peak prefill
    11K tok/s
    by @Sandberg
  • gemma-4-12B-it-qat-w4a16-ctRTX 3090 24GB · vllm 0.32.0 · W4A166 contexts × 8 concurrency levels
    Peak decode
    1,499.1 tok/s
    Peak prefill
    3,276 tok/s
    by @Sandberg
  • gemma-3-12b-it-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    1,335.5 tok/s
    Peak prefill
    4,882 tok/s
    by @Sandberg
  • Agents-A1-4B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,155.8 tok/s
    Peak prefill
    11.7K tok/s
    by @Sandberg
  • gemma-4-12B-it-FP8-DynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,388.8 tok/s
    Peak prefill
    3,857 tok/s
    by @Sandberg
  • Qwen3-8B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
    Peak decode
    1,889.2 tok/s
    Peak prefill
    5,567 tok/s
    by @Sandberg
  • Qwen3.5-4B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,189.3 tok/s
    Peak prefill
    11.1K tok/s
    by @Sandberg
  • Qwen3.5-9B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,915.2 tok/s
    Peak prefill
    6,700 tok/s
    by @Sandberg
  • deepseek-r1-distill-qwen-14b-awqRTX 3090 24GB · vllm 0.32.0 · AWQ4 contexts × 8 concurrency levels
    Peak decode
    291.6 tok/s
    Peak prefill
    3,188 tok/s
    by @Sandberg
  • Qwen3.5-4B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,186.7 tok/s
    Peak prefill
    11.8K tok/s
    by @Sandberg
  • Qwen3-14B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP83 contexts × 8 concurrency levels
    Peak decode
    1,436.5 tok/s
    Peak prefill
    3,122 tok/s
    by @Sandberg
  • Qwen3.5-9B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    3,499.4 tok/s
    Peak prefill
    6,902 tok/s
    by @Sandberg
  • Qwen3-14B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    546.1 tok/s
    Peak prefill
    3,243 tok/s
    by @Sandberg
  • Trinity-Mini-FP8-BlockCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 block5 contexts × 8 concurrency levels
    Peak decode
    2,422.8 tok/s
    Peak prefill
    10.3K tok/s
    by @Sandberg
  • Ministral-3-3B-Instruct-2512-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,524.0 tok/s
    Peak prefill
    12.3K tok/s
    by @Sandberg