Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 3 of 45.
  • Fara1.5-4B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,155.1 tok/s
    Peak prefill
    10.8K tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,152.0 tok/s
    Peak prefill
    2,707 tok/s
    by @Sandberg
  • DeepSeek-V4.1-Flash-1017RTX PRO 6000 Blackwell Server Edition 96GB ×4 · vllm 0.33.0rc1.dev141 · MXFP4+FP87 contexts × 8 concurrency levels
    Peak decode
    1,198.5 tok/s
    Peak prefill
    19.3K tok/s
    by @keennay
  • Agents-A1-4B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,124.5 tok/s
    Peak prefill
    10.8K tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    737.3 tok/s
    Peak prefill
    2,557 tok/s
    by @Sandberg
  • Qwen3.5-4B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,099.1 tok/s
    Peak prefill
    10.3K tok/s
    by @Sandberg
  • Hy-MT2-7B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    573.6 tok/s
    Peak prefill
    6,528 tok/s
    by @Sandberg
  • Qwen3-8B-AWQRTX 3060 12GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    218.2 tok/s
    Peak prefill
    1,481 tok/s
    by @Sandberg
  • Ornith-1.5-9B-NVFP4RTX 3060 12GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    367.0 tok/s
    Peak prefill
    1,594 tok/s
    by @Sandberg
  • Nanbeige4.3-33B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP87 contexts × 8 concurrency levels
    Peak decode
    612.9 tok/s
    Peak prefill
    1,015 tok/s
    by @Sandberg
  • Llama-3.3-70B-Instruct-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
    Peak decode
    245.8 tok/s
    Peak prefill
    477.0 tok/s
    by @Sandberg
  • Qwen3.5-4B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    2,152.1 tok/s
    Peak prefill
    10.7K tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit3 contexts × 8 concurrency levels
    Peak decode
    144.6 tok/s
    Peak prefill
    1,034 tok/s
    by @Sandberg
  • Llama-3.2-3B-Instruct-FP8-dynamicRTX 3060 12GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    664.2 tok/s
    Peak prefill
    1,840 tok/s
    by @Sandberg
  • Ministral-3-3B-Instruct-2512-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,922.0 tok/s
    Peak prefill
    11.4K tok/s
    by @Sandberg
  • granite-4.2-30b-fp8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP85 contexts × 8 concurrency levels
    Peak decode
    510.3 tok/s
    Peak prefill
    2,016 tok/s
    by @Sandberg
  • Tess-4-9BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,184.7 tok/s
    Peak prefill
    6,499 tok/s
    by @Sandberg
  • Qwen3-8B-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    570.8 tok/s
    Peak prefill
    5,348 tok/s
    by @Sandberg
  • granite-4.2-8b-nvfp4RTX 3060 12GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    204.2 tok/s
    Peak prefill
    1,302 tok/s
    by @Sandberg
  • granite-4.2-3b-nvfp4RTX 3060 12GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    1,146.0 tok/s
    Peak prefill
    2,797 tok/s
    by @Sandberg
  • InternVL3-38B-FP8-DynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic3 contexts × 8 concurrency levels
    Peak decode
    138.5 tok/s
    Peak prefill
    1,846 tok/s
    by @Sandberg
  • granite-4.2-8b-fp8RTX 3090 24GB · vllm 0.32.0 · FP85 contexts × 8 concurrency levels
    Peak decode
    1,091.3 tok/s
    Peak prefill
    5,380 tok/s
    by @Sandberg
  • granite-4.2-3b-fp8RTX 3060 12GB · vllm 0.32.0 · FP85 contexts × 8 concurrency levels
    Peak decode
    146.4 tok/s
    Peak prefill
    2,298 tok/s
    by @Sandberg
  • DeepSeek-R1-Distill-Qwen-32B-FP8-dynamicCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    509.1 tok/s
    Peak prefill
    1,841 tok/s
    by @Sandberg
  • Qwen3-1.7BRTX 3060 12GB · vllm 0.32.0 · BF162 contexts × 8 concurrency levels
    Peak decode
    979.3 tok/s
    Peak prefill
    2,788 tok/s
    by @Sandberg