Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 4 of 45.
  • Tess-4-9BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    435.7 tok/s
    Peak prefill
    5,762 tok/s
    by @Sandberg
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3060 12GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    625.4 tok/s
    Peak prefill
    9,918 tok/s
    by @Sandberg
  • Qwen3-0.6BRTX 3060 12GB · vllm 0.32.0 · BF163 contexts × 8 concurrency levels
    Peak decode
    625.6 tok/s
    Peak prefill
    7,105 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    1,682.1 tok/s
    Peak prefill
    5,757 tok/s
    by @Sandberg
  • GLM-4.7-218B-W4A16CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · AutoRound W4A163 contexts × 8 concurrency levels
    Peak decode
    88.5 tok/s
    Peak prefill
    593.0 tok/s
    by @Sandberg
  • Tess-4-27B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    163.7 tok/s
    Peak prefill
    2,110 tok/s
    by @Sandberg
  • gemma-3-12b-it-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    745.1 tok/s
    Peak prefill
    4,370 tok/s
    by @Sandberg
  • Step3-VL-10BRTX 3090 24GB · vllm 0.32.0 · BF164 contexts × 8 concurrency levels
    Peak decode
    398.9 tok/s
    Peak prefill
    7,033 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit5 contexts × 8 concurrency levels
    Peak decode
    189.0 tok/s
    Peak prefill
    950.0 tok/s
    by @Sandberg
  • Qwen3-Coder-30B-A3B-Instruct-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit4 contexts × 8 concurrency levels
    Peak decode
    780.8 tok/s
    Peak prefill
    9,630 tok/s
    by @Sandberg
  • Qwen3.5-9B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic2 contexts × 8 concurrency levels
    Peak decode
    1,134.8 tok/s
    Peak prefill
    6,096 tok/s
    by @Sandberg
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1284 contexts × 8 concurrency levels
    Peak decode
    329.4 tok/s
    Peak prefill
    2,163 tok/s
    by @Sandberg
  • Ornith-1.5-9B-NVFP4RTX 3060 12GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    599.9 tok/s
    Peak prefill
    1,463 tok/s
    by @Sandberg
  • Qwen3.5-9B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    1,561.1 tok/s
    Peak prefill
    7,124 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3060 12GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
    Peak decode
    470.7 tok/s
    Peak prefill
    1,240 tok/s
    by @Sandberg
  • Qwen3.5-9B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    1,848.4 tok/s
    Peak prefill
    6,418 tok/s
    by @Sandberg
  • granite-4.2-8b-nvfp4RTX 3060 12GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    311.6 tok/s
    Peak prefill
    910.0 tok/s
    by @Sandberg
  • Qwen3.5-27B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit3 contexts × 8 concurrency levels
    Peak decode
    163.3 tok/s
    Peak prefill
    2,141 tok/s
    by @Sandberg
  • Mistral-Medium-3.5-128B-NVFP4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP4 mixed6 contexts × 8 concurrency levels
    Peak decode
    234.7 tok/s
    Peak prefill
    370.0 tok/s
    by @Sandberg
  • Qwen3-14B-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    205.6 tok/s
    Peak prefill
    3,290 tok/s
    by @Sandberg
  • Qwen3-8B-AWQRTX 3060 12GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    332.8 tok/s
    Peak prefill
    1,021 tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,279.8 tok/s
    Peak prefill
    2,343 tok/s
    by @Sandberg
  • Ornith-1.5-9BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,186.0 tok/s
    Peak prefill
    6,398 tok/s
    by @Sandberg
  • Ornith-1.5-9BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    473.2 tok/s
    Peak prefill
    5,717 tok/s
    by @Sandberg
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3060 12GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    1,273.2 tok/s
    Peak prefill
    2,347 tok/s
    by @Sandberg