Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 11 of 45.
  • gemma-4-12B-it-FP8-DynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    935.4 tok/s
    Peak prefill
    3,847 tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    2,080.9 tok/s
    Peak prefill
    9,922 tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    3,263.4 tok/s
    Peak prefill
    10.6K tok/s
    by @Sandberg
  • Phi-4-reasoning-plus-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    204.6 tok/s
    Peak prefill
    3,317 tok/s
    by @Sandberg
  • Tess-4-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    672.0 tok/s
    Peak prefill
    2,210 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    2,729.1 tok/s
    Peak prefill
    8,877 tok/s
    by @Sandberg
  • granite-4.2-3b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    3,409.4 tok/s
    Peak prefill
    4,722 tok/s
    by @Sandberg
  • gpt-oss-20bRTX 3090 24GB · vllm 0.32.0 · MXFP45 contexts × 8 concurrency levels
    Peak decode
    2,708.2 tok/s
    Peak prefill
    6,871 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP48 contexts × 8 concurrency levels
    Peak decode
    1,736.7 tok/s
    Peak prefill
    7,524 tok/s
    by @Sandberg
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,095.8 tok/s
    Peak prefill
    8,943 tok/s
    by @Sandberg
  • Qwen3.5-27B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit4 contexts × 8 concurrency levels
    Peak decode
    180.5 tok/s
    Peak prefill
    1,804 tok/s
    by @Sandberg
  • Tess-4-27B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    187.2 tok/s
    Peak prefill
    1,483 tok/s
    by @Sandberg
  • Fara1.5-27B-AWQ-W4A16RTX 3090 24GB · vllm 0.32.0 · AWQ W4A165 contexts × 8 concurrency levels
    Peak decode
    366.7 tok/s
    Peak prefill
    1,808 tok/s
    by @Sandberg
  • Sage-A1.5-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    3,054.1 tok/s
    Peak prefill
    7,513 tok/s
    by @Sandberg
  • Qwen3-Coder-30B-A3B-Instruct-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    985.1 tok/s
    Peak prefill
    4,078 tok/s
    by @Sandberg
  • gemma-4-31B-it-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    557.2 tok/s
    Peak prefill
    1,519 tok/s
    by @Sandberg
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1285 contexts × 8 concurrency levels
    Peak decode
    391.8 tok/s
    Peak prefill
    1,500 tok/s
    by @Sandberg
  • granite-4.2-30b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    152.7 tok/s
    Peak prefill
    1,365 tok/s
    by @Sandberg
  • Llama-3.3-70B-Instruct-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
    Peak decode
    165.0 tok/s
    Peak prefill
    702.0 tok/s
    by @Sandberg
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    993.9 tok/s
    Peak prefill
    9,721 tok/s
    by @Sandberg
  • granite-4.2-3b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    1,719.6 tok/s
    Peak prefill
    10.8K tok/s
    by @Sandberg
  • Fara1.5-27B-AWQ-W4A16RTX 3090 24GB · vllm 0.32.0 · AWQ W4A163 contexts × 8 concurrency levels
    Peak decode
    170.6 tok/s
    Peak prefill
    2,162 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    364.7 tok/s
    Peak prefill
    8,151 tok/s
    by @Sandberg
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1284 contexts × 8 concurrency levels
    Peak decode
    120.9 tok/s
    Peak prefill
    1,749 tok/s
    by @Sandberg
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,671.4 tok/s
    Peak prefill
    6,435 tok/s
    by @Sandberg