Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 351

Submitted by @Sandberg. Newest first. Page 14 of 15.
  • Qwen3.5-4B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    3,451.6 tok/s
    Peak prefill
    9,663 tok/s
  • deepseek-r1-distill-qwen-14b-awqCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ5 contexts × 8 concurrency levels
    Peak decode
    1,487.7 tok/s
    Peak prefill
    3,427 tok/s
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP44 contexts × 8 concurrency levels
    Peak decode
    264.2 tok/s
    Peak prefill
    8,160 tok/s
  • Qwen3.6-35B-A3B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    2,916.2 tok/s
    Peak prefill
    6,639 tok/s
  • Phi-4-reasoning-plus-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP43 contexts × 8 concurrency levels
    Peak decode
    1,489.8 tok/s
    Peak prefill
    3,267 tok/s
  • granite-4.2-3b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    1,775.8 tok/s
    Peak prefill
    12.1K tok/s
  • Ornith-1.5-9B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    3,578.1 tok/s
    Peak prefill
    5,826 tok/s
  • Qwen3.6-35B-A3B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    3,006.5 tok/s
    Peak prefill
    7,043 tok/s
  • granite-4.2-8b-nvfp4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP45 contexts × 8 concurrency levels
    Peak decode
    1,825.1 tok/s
    Peak prefill
    5,123 tok/s
  • Hy-MT2-30B-A3B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    1,258.7 tok/s
    Peak prefill
    7,809 tok/s
  • Fara1.5-4B-AutoRound-W4A16-RTNCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    4,682.7 tok/s
    Peak prefill
    9,791 tok/s
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP48 contexts × 8 concurrency levels
    Peak decode
    2,385.6 tok/s
    Peak prefill
    7,570 tok/s
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP48 contexts × 8 concurrency levels
    Peak decode
    1,604.3 tok/s
    Peak prefill
    9,619 tok/s
  • Qwen3.5-4B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    4,810.5 tok/s
    Peak prefill
    9,988 tok/s
  • Qwen3.8-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    652.0 tok/s
    Peak prefill
    1,688 tok/s
  • Qwen3.8-27B-FP8CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    821.9 tok/s
    Peak prefill
    1,717 tok/s
  • Qwen3.8-27B-INT4RTX 3090 24GB · vllm 0.32.0 · INT4 W4A16 g1283 contexts × 8 concurrency levels
    Peak decode
    171.1 tok/s
    Peak prefill
    2,179 tok/s
  • gpt-oss-20bRTX 3090 24GB · vllm 0.32.0 · MXFP45 contexts × 8 concurrency levels
    Peak decode
    1,813.0 tok/s
    Peak prefill
    10.5K tok/s
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP48 contexts × 8 concurrency levels
    Peak decode
    2,954.9 tok/s
    Peak prefill
    8,465 tok/s
  • Tess-4-27B-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,101.0 tok/s
    Peak prefill
    1,837 tok/s
  • gemma-4-31B-it-qat-w4a16-ctCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · W4A166 contexts × 8 concurrency levels
    Peak decode
    683.8 tok/s
    Peak prefill
    1,368 tok/s
  • gpt-oss-20bCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · MXFP45 contexts × 8 concurrency levels
    Peak decode
    3,816.0 tok/s
    Peak prefill
    8,773 tok/s
  • GLM-5.3-Flash-W4A16-MTPCMP 170HX 64GB ×4 · vllm 0.32.0rc1.dev215+g819146023 · W4A167 contexts × 5 concurrency levels
    Peak decode
    281.9 tok/s
    Peak prefill
    6,577 tok/s
  • Qwen3.8-27B-INT4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · INT4 W4A16 g1286 contexts × 8 concurrency levels
    Peak decode
    864.5 tok/s
    Peak prefill
    1,922 tok/s
  • Qwen3.8-27B-INT4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · INT4 W4A16 g1286 contexts × 8 concurrency levels
    Peak decode
    1,247.4 tok/s
    Peak prefill
    1,949 tok/s