Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 13 of 45.
  • GLM-5.3-569B-W4A16CMP 170HX 64GB ×8 · vllm 0.30.1rc1.dev379+gab60b723a · W4A168 contexts × 5 concurrency levels
    Peak decode
    311.9 tok/s
    Peak prefill
    3,678 tok/s
    by @Sandberg
  • deepseek-r1-distill-qwen-14b-awqRTX 3090 24GB · vllm 0.32.0 · AWQ4 contexts × 8 concurrency levels
    Peak decode
    298.0 tok/s
    Peak prefill
    3,722 tok/s
    by @Sandberg
  • Qwen3.5-27B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,140.7 tok/s
    Peak prefill
    1,976 tok/s
    by @Sandberg
  • Qwen3-14B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    557.0 tok/s
    Peak prefill
    3,776 tok/s
    by @Sandberg
  • Phi-4-reasoning-plus-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    587.4 tok/s
    Peak prefill
    3,788 tok/s
    by @Sandberg
  • Qwen3.5-27B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,001.0 tok/s
    Peak prefill
    1,921 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-12B-v2-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit5 contexts × 8 concurrency levels
    Peak decode
    1,369.5 tok/s
    Peak prefill
    4,527 tok/s
    by @Sandberg
  • Ornith-1.5-9B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP46 contexts × 8 concurrency levels
    Peak decode
    2,997.4 tok/s
    Peak prefill
    6,712 tok/s
    by @Sandberg
  • Qwen3-Coder-30B-A3B-Instruct-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,819.0 tok/s
    Peak prefill
    7,957 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-Nano-9B-v2-quantized.w4a16RTX 3090 24GB · vllm 0.32.0 · W4A165 contexts × 8 concurrency levels
    Peak decode
    1,780.0 tok/s
    Peak prefill
    5,646 tok/s
    by @Sandberg
  • Fara1.5-27B-AWQ-W4A16CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ W4A166 contexts × 8 concurrency levels
    Peak decode
    1,170.9 tok/s
    Peak prefill
    1,967 tok/s
    by @Sandberg
  • granite-4.2-8b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    708.5 tok/s
    Peak prefill
    5,842 tok/s
    by @Sandberg
  • GLM-5.3-569B-W4A16CMP 170HX 64GB ×8 · vllm 0.30.1rc1.dev379+gab60b723a · W4A168 contexts × 5 concurrency levels
    Peak decode
    229.8 tok/s
    Peak prefill
    3,732 tok/s
    by @Sandberg
  • gemma-4-12B-it-qat-w4a16-ctCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · W4A166 contexts × 8 concurrency levels
    Peak decode
    2,359.3 tok/s
    Peak prefill
    3,489 tok/s
    by @Sandberg
  • Qwen3-8B-AWQRTX 3090 24GB · vllm 0.32.0 · AWQ3 contexts × 8 concurrency levels
    Peak decode
    751.7 tok/s
    Peak prefill
    6,660 tok/s
    by @Sandberg
  • Fara1.5-4B-AutoRound-W4A16-RTNRTX 3090 24GB · vllm 0.32.0 · AutoRound W4A166 contexts × 8 concurrency levels
    Peak decode
    3,297.1 tok/s
    Peak prefill
    11.8K tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    2,170.5 tok/s
    Peak prefill
    11K tok/s
    by @Sandberg
  • DeepSeek-V4.1-Flash-1017CMP 170HX 64GB ×4 · vllm 0.32.0rc1.dev586+ga0c57b523 · EXL36 contexts × 5 concurrency levels
    Peak decode
    221.3 tok/s
    Peak prefill
    1,483 tok/s
    by @Sandberg
  • Qwen3.8-Flash-Next-W4A16-AutoRoundCMP 170HX 64GB ×2 · vllm 0.30.0 · W4A166 contexts × 8 concurrency levels
    Peak decode
    313.0 tok/s
    Peak prefill
    6,135 tok/s
    by @Sandberg
  • Qwen3.5-4B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    3,391.0 tok/s
    Peak prefill
    11.8K tok/s
    by @Sandberg
  • granite-4.2-3b-nvfp4RTX 3090 24GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    3,673.7 tok/s
    Peak prefill
    5,452 tok/s
    by @Sandberg
  • DeepSeek-V4.1-Flash-1017CMP 170HX 64GB ×4 · vllm 0.32.0rc1.dev586+ga0c57b523 · EXL36 contexts × 5 concurrency levels
    Peak decode
    255.6 tok/s
    Peak prefill
    1,531 tok/s
    by @Sandberg
  • gpt-oss-20bRTX 3090 24GB · vllm 0.32.0 · MXFP45 contexts × 8 concurrency levels
    Peak decode
    2,849.9 tok/s
    Peak prefill
    7,663 tok/s
    by @Sandberg
  • Qwen3.8-Flash-Next-exl3CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · EXL3 5.05 bpw6 contexts × 8 concurrency levels
    Peak decode
    287.6 tok/s
    Peak prefill
    1,462 tok/s
    by @Sandberg
  • NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP48 contexts × 8 concurrency levels
    Peak decode
    1,784.8 tok/s
    Peak prefill
    8,347 tok/s
    by @Sandberg