Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 351

Submitted by @Sandberg. Newest first. Page 12 of 15.
  • NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3090 24GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
    Peak decode
    1,768.0 tok/s
    Peak prefill
    5,097 tok/s
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    3,233.2 tok/s
    Peak prefill
    9,370 tok/s
  • MiniCPM5-1B-exl3CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · EXL3 3.00 bpw5 contexts × 8 concurrency levels
    Peak decode
    3,349.3 tok/s
    Peak prefill
    5,883 tok/s
  • granite-4.2-8b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    1,640.0 tok/s
    Peak prefill
    3,402 tok/s
  • Step-3.7-Flash-148BCMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    781.0 tok/s
    Peak prefill
    5,092 tok/s
  • Qwen3-8B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    1,749.9 tok/s
    Peak prefill
    3,824 tok/s
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3090 24GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    3,981.6 tok/s
    Peak prefill
    9,375 tok/s
  • Qwen3-235B-A22B-Instruct-2507-REAP-nvfp4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
    Peak decode
    175.4 tok/s
    Peak prefill
    2,357 tok/s
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    4,063.4 tok/s
    Peak prefill
    9,313 tok/s
  • MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
    Peak decode
    274.9 tok/s
    Peak prefill
    4,025 tok/s
  • GLM-5.2-W4A16-MTPCMP 170HX 64GB ×8 · vllm 0.30.1rc1.dev379+gab60b723a · W4A165 contexts × 8 concurrency levels
    Peak decode
    297.2 tok/s
    Peak prefill
    3,369 tok/s
  • Qwen3.5-122B-A10B-GPTQ-Int4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · GPTQ Int46 contexts × 8 concurrency levels
    Peak decode
    1,006.6 tok/s
    Peak prefill
    5,549 tok/s
  • gemma-4-26B-A4B-it-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    650.1 tok/s
    Peak prefill
    7,754 tok/s
  • gemma-4-12B-it-qat-w4a16-ctRTX 3090 24GB · vllm 0.32.0 · W4A166 contexts × 8 concurrency levels
    Peak decode
    1,550.1 tok/s
    Peak prefill
    3,752 tok/s
  • Qwen3.5-122B-A10B-GPTQ-Int4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · GPTQ Int46 contexts × 8 concurrency levels
    Peak decode
    1,628.1 tok/s
    Peak prefill
    5,800 tok/s
  • GLM-5.3-569B-W4A16CMP 170HX 64GB ×8 · vllm 0.30.1rc1.dev379+gab60b723a · W4A168 contexts × 5 concurrency levels
    Peak decode
    311.9 tok/s
    Peak prefill
    3,678 tok/s
  • deepseek-r1-distill-qwen-14b-awqRTX 3090 24GB · vllm 0.32.0 · AWQ4 contexts × 8 concurrency levels
    Peak decode
    298.0 tok/s
    Peak prefill
    3,722 tok/s
  • Qwen3.5-27B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,140.7 tok/s
    Peak prefill
    1,976 tok/s
  • Qwen3-14B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    557.0 tok/s
    Peak prefill
    3,776 tok/s
  • Phi-4-reasoning-plus-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    587.4 tok/s
    Peak prefill
    3,788 tok/s
  • Qwen3.5-27B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,001.0 tok/s
    Peak prefill
    1,921 tok/s
  • NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit5 contexts × 8 concurrency levels
    Peak decode
    1,369.5 tok/s
    Peak prefill
    4,527 tok/s
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    2,997.4 tok/s
    Peak prefill
    6,712 tok/s
  • Qwen3-Coder-30B-A3B-Instruct-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,819.0 tok/s
    Peak prefill
    7,957 tok/s
  • NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3090 24GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
    Peak decode
    1,780.0 tok/s
    Peak prefill
    5,646 tok/s