Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 6 of 45.
  • NVIDIA-Nemotron-3-Nano-4B-BF16RTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    2,579.8 tok/s
    Peak prefill
    14K tok/s
    by @Sandberg
  • Fara1.5-4B-FP8RTX 3060 12GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    909.6 tok/s
    Peak prefill
    2,375 tok/s
    by @Sandberg
  • Qwen3-8BRTX 3090 24GB · vllm 0.32.0 · BF163 contexts × 8 concurrency levels
    Peak decode
    396.4 tok/s
    Peak prefill
    7,042 tok/s
    by @Sandberg
  • Llama-3.3-70B-Instruct-FP8-dynamicCMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    234.7 tok/s
    Peak prefill
    1,356 tok/s
    by @Sandberg
  • granite-4.2-30b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    278.4 tok/s
    Peak prefill
    1,173 tok/s
    by @Sandberg
  • DeepSeek-R1-Distill-Qwen-14B-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    338.4 tok/s
    Peak prefill
    2,137 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-3-Nano-4B-FP8RTX 3060 12GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    952.3 tok/s
    Peak prefill
    3,010 tok/s
    by @Sandberg
  • gpt-oss-120bCMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · MXFP45 contexts × 8 concurrency levels
    Peak decode
    2,009.9 tok/s
    Peak prefill
    7,548 tok/s
    by @Sandberg
  • Hy-MT2-7B-FP8RTX 3090 24GB · vllm 0.32.0 · FP85 contexts × 8 concurrency levels
    Peak decode
    239.1 tok/s
    Peak prefill
    6,574 tok/s
    by @Sandberg
  • Fara1.5-27B-AWQ-W4A16RTX 3090 24GB · vllm 0.32.0 · AWQ W4A165 contexts × 8 concurrency levels
    Peak decode
    394.4 tok/s
    Peak prefill
    1,537 tok/s
    by @Sandberg
  • Llama-3.2-3B-Instruct-FP8-dynamicRTX 3060 12GB · vllm 0.32.0 · FP8 dynamic4 contexts × 8 concurrency levels
    Peak decode
    267.2 tok/s
    Peak prefill
    3,265 tok/s
    by @Sandberg
  • Step3-VL-10BRTX 3090 24GB · vllm 0.32.0 · BF163 contexts × 8 concurrency levels
    Peak decode
    200.5 tok/s
    Peak prefill
    6,946 tok/s
    by @Sandberg
  • granite-4.2-3b-fp8RTX 3060 12GB · vllm 0.32.0 · FP84 contexts × 8 concurrency levels
    Peak decode
    478.0 tok/s
    Peak prefill
    2,841 tok/s
    by @Sandberg
  • Ornith-1.5-9BRTX 3090 24GB · vllm 0.32.0 · BF163 contexts × 8 concurrency levels
    Peak decode
    259.9 tok/s
    Peak prefill
    6,640 tok/s
    by @Sandberg
  • Qwen3-1.7B-FP8RTX 3060 12GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    564.3 tok/s
    Peak prefill
    5,516 tok/s
    by @Sandberg
  • Qwen3-0.6B-FP8RTX 3060 12GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    665.4 tok/s
    Peak prefill
    9,704 tok/s
    by @Sandberg
  • Ornith-1.5-9BRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    378.8 tok/s
    Peak prefill
    6,966 tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    678.0 tok/s
    Peak prefill
    2,654 tok/s
    by @Sandberg
  • Qwen3-Coder-30B-A3B-Instruct-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    933.6 tok/s
    Peak prefill
    3,531 tok/s
    by @Sandberg
  • gemma-4-E4B-itRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    563.3 tok/s
    Peak prefill
    10.2K tok/s
    by @Sandberg
  • Ornith-1.5-9B-NVFP4RTX 3060 12GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    367.1 tok/s
    Peak prefill
    1,599 tok/s
    by @Sandberg
  • Qwen3.5-4BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    2,039.0 tok/s
    Peak prefill
    11.1K tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3060 12GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
    Peak decode
    480.5 tok/s
    Peak prefill
    1,337 tok/s
    by @Sandberg
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    974.8 tok/s
    Peak prefill
    9,139 tok/s
    by @Sandberg
  • Qwen3.5-4BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,836.2 tok/s
    Peak prefill
    11.8K tok/s
    by @Sandberg