Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 7 of 45.
  • Qwen3.5-4B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    812.4 tok/s
    Peak prefill
    2,765 tok/s
    by @Sandberg
  • granite-4.2-3bRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    1,222.5 tok/s
    Peak prefill
    12.7K tok/s
    by @Sandberg
  • DeepSeek-R1-Distill-Qwen-14B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    844.7 tok/s
    Peak prefill
    3,910 tok/s
    by @Sandberg
  • MiniCPM5-1BRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    4,618.6 tok/s
    Peak prefill
    36.8K tok/s
    by @Sandberg
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3060 12GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    820.4 tok/s
    Peak prefill
    2,790 tok/s
    by @Sandberg
  • granite-4.2-3b-nvfp4RTX 3060 12GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    591.9 tok/s
    Peak prefill
    2,775 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP4 mixed6 contexts × 8 concurrency levels
    Peak decode
    1,078.6 tok/s
    Peak prefill
    3,902 tok/s
    by @Sandberg
  • Qwen3-Next-80B-A3B-Thinking-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    2,121.7 tok/s
    Peak prefill
    7,388 tok/s
    by @Sandberg
  • Ornith-1.5-35B-A3B-W4A16-SYMCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · W4A166 contexts × 8 concurrency levels
    Peak decode
    3,131.6 tok/s
    Peak prefill
    7,431 tok/s
    by @Sandberg
  • gpt-oss-20bCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · MXFP45 contexts × 8 concurrency levels
    Peak decode
    3,766.5 tok/s
    Peak prefill
    8,710 tok/s
    by @Sandberg
  • Muse-Glimmer-30B-FP8-blockCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 block5 contexts × 8 concurrency levels
    Peak decode
    1,279.7 tok/s
    Peak prefill
    2,288 tok/s
    by @Sandberg
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1284 contexts × 8 concurrency levels
    Peak decode
    120.1 tok/s
    Peak prefill
    1,483 tok/s
    by @Sandberg
  • Qwen3.8-27B-INT8-W8A16-MTPCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · INT8 W8A166 contexts × 8 concurrency levels
    Peak decode
    616.4 tok/s
    Peak prefill
    1,633 tok/s
    by @Sandberg
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,704.7 tok/s
    Peak prefill
    5,739 tok/s
    by @Sandberg
  • Qwen3.8-27B-INT8-W8A16-MTPCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · INT8 W8A166 contexts × 8 concurrency levels
    Peak decode
    766.4 tok/s
    Peak prefill
    1,664 tok/s
    by @Sandberg
  • gemma-4-26B-A4B-it-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,092.4 tok/s
    Peak prefill
    5,438 tok/s
    by @Sandberg
  • gemma-4-12B-it-qat-w4a16-ctRTX 3090 24GB · vllm 0.32.0 · W4A166 contexts × 8 concurrency levels
    Peak decode
    1,326.8 tok/s
    Peak prefill
    2,690 tok/s
    by @Sandberg
  • deepseek-r1-distill-qwen-14b-awqRTX 3090 24GB · vllm 0.32.0 · AWQ5 contexts × 8 concurrency levels
    Peak decode
    742.0 tok/s
    Peak prefill
    1,839 tok/s
    by @Sandberg
  • Qwen3-14B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    1,325.8 tok/s
    Peak prefill
    1,999 tok/s
    by @Sandberg
  • Phi-4-reasoning-plus-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    779.6 tok/s
    Peak prefill
    2,035 tok/s
    by @Sandberg
  • LFM3-1.7B-TT-Mixed-BFP4-BFP8-P150Tenstorrent P150 32GB · vllm 0.1.dev14187+g03fa3af2e · BFP46 contexts × 5 concurrency levels
    Peak decode
    2,892.8 tok/s
    Peak prefill
    47.8K tok/s
    by @Lottolabs
  • NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit5 contexts × 8 concurrency levels
    Peak decode
    1,469.4 tok/s
    Peak prefill
    3,587 tok/s
    by @Sandberg
  • Tess-4-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    837.8 tok/s
    Peak prefill
    2,161 tok/s
    by @Sandberg
  • internlm_internlm3-8b-instruct-autoround-int8-gs128-symCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AutoRound INT8 W8A16 gs1283 contexts × 8 concurrency levels
    Peak decode
    4,081.9 tok/s
    Peak prefill
    4,878 tok/s
    by @Sandberg
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    3,075.4 tok/s
    Peak prefill
    5,390 tok/s
    by @Sandberg