Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 12 of 45.
  • gemma-4-26B-A4B-it-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,137.8 tok/s
    Peak prefill
    5,892 tok/s
    by @Sandberg
  • gemma-4-12B-it-qat-w4a16-ctRTX 3090 24GB · vllm 0.32.0 · W4A166 contexts × 8 concurrency levels
    Peak decode
    1,397.0 tok/s
    Peak prefill
    3,012 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP48 contexts × 8 concurrency levels
    Peak decode
    1,791.8 tok/s
    Peak prefill
    8,758 tok/s
    by @Sandberg
  • deepseek-r1-distill-qwen-14b-awqRTX 3090 24GB · vllm 0.32.0 · AWQ5 contexts × 8 concurrency levels
    Peak decode
    777.1 tok/s
    Peak prefill
    2,138 tok/s
    by @Sandberg
  • Qwen3-14B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    1,407.4 tok/s
    Peak prefill
    2,331 tok/s
    by @Sandberg
  • Phi-4-reasoning-plus-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    814.6 tok/s
    Peak prefill
    2,365 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit5 contexts × 8 concurrency levels
    Peak decode
    1,551.0 tok/s
    Peak prefill
    4,074 tok/s
    by @Sandberg
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1283 contexts × 8 concurrency levels
    Peak decode
    167.0 tok/s
    Peak prefill
    1,845 tok/s
    by @Sandberg
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    3,274.3 tok/s
    Peak prefill
    6,070 tok/s
    by @Sandberg
  • gpt-oss-20bRTX 3090 24GB · vllm 0.32.0 · MXFP45 contexts × 8 concurrency levels
    Peak decode
    1,766.5 tok/s
    Peak prefill
    9,483 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3090 24GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
    Peak decode
    1,768.0 tok/s
    Peak prefill
    5,097 tok/s
    by @Sandberg
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    3,233.2 tok/s
    Peak prefill
    9,370 tok/s
    by @Sandberg
  • MiniCPM5-1B-exl3CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · EXL3 3.00 bpw5 contexts × 8 concurrency levels
    Peak decode
    3,349.3 tok/s
    Peak prefill
    5,883 tok/s
    by @Sandberg
  • granite-4.2-8b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    1,640.0 tok/s
    Peak prefill
    3,402 tok/s
    by @Sandberg
  • Step-3.7-Flash-148BCMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    781.0 tok/s
    Peak prefill
    5,092 tok/s
    by @Sandberg
  • Qwen3-8B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    1,749.9 tok/s
    Peak prefill
    3,824 tok/s
    by @Sandberg
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3090 24GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    3,981.6 tok/s
    Peak prefill
    9,375 tok/s
    by @Sandberg
  • Qwen3-235B-A22B-Instruct-2507-REAP-nvfp4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
    Peak decode
    175.4 tok/s
    Peak prefill
    2,357 tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    4,063.4 tok/s
    Peak prefill
    9,313 tok/s
    by @Sandberg
  • MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
    Peak decode
    274.9 tok/s
    Peak prefill
    4,025 tok/s
    by @Sandberg
  • GLM-5.2-W4A16-MTPCMP 170HX 64GB ×8 · vllm 0.30.1rc1.dev379+gab60b723a · W4A165 contexts × 8 concurrency levels
    Peak decode
    297.2 tok/s
    Peak prefill
    3,369 tok/s
    by @Sandberg
  • Qwen3.5-122B-A10B-GPTQ-Int4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · GPTQ Int46 contexts × 8 concurrency levels
    Peak decode
    1,006.6 tok/s
    Peak prefill
    5,549 tok/s
    by @Sandberg
  • gemma-4-26B-A4B-it-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    650.1 tok/s
    Peak prefill
    7,754 tok/s
    by @Sandberg
  • gemma-4-12B-it-qat-w4a16-ctRTX 3090 24GB · vllm 0.32.0 · W4A166 contexts × 8 concurrency levels
    Peak decode
    1,550.1 tok/s
    Peak prefill
    3,752 tok/s
    by @Sandberg
  • Qwen3.5-122B-A10B-GPTQ-Int4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · GPTQ Int46 contexts × 8 concurrency levels
    Peak decode
    1,628.1 tok/s
    Peak prefill
    5,800 tok/s
    by @Sandberg