Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

All runs 1,102

Newest first. Page 5 of 45.
  • GLM-4.7-185B-W4A16CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · GPTQ W4A166 contexts × 8 concurrency levels
    Peak decode
    183.9 tok/s
    Peak prefill
    596.0 tok/s
    by @Sandberg
  • Ministral-3-3B-Instruct-2512-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,988.0 tok/s
    Peak prefill
    12.4K tok/s
    by @Sandberg
  • granite-4.2-3b-nvfp4RTX 3060 12GB · vllm 0.32.0 · NVFP45 contexts × 8 concurrency levels
    Peak decode
    1,305.8 tok/s
    Peak prefill
    1,611 tok/s
    by @Sandberg
  • granite-4.2-8b-fp8RTX 3090 24GB · vllm 0.32.0 · FP85 contexts × 8 concurrency levels
    Peak decode
    1,105.2 tok/s
    Peak prefill
    6,099 tok/s
    by @Sandberg
  • granite-4.2-8bRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    316.4 tok/s
    Peak prefill
    3,518 tok/s
    by @Sandberg
  • gemma-4-12B-it-FP8-DynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    923.8 tok/s
    Peak prefill
    3,385 tok/s
    by @Sandberg
  • gemma-4-E4B-itRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    1,770.8 tok/s
    Peak prefill
    10.1K tok/s
    by @Sandberg
  • Phi-4-reasoning-plus-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    202.6 tok/s
    Peak prefill
    3,082 tok/s
    by @Sandberg
  • gemma-3-12b-it-FP8-dynamicRTX 3090 24GB · vllm 0.32.0 · FP8 dynamic5 contexts × 8 concurrency levels
    Peak decode
    759.4 tok/s
    Peak prefill
    4,826 tok/s
    by @Sandberg
  • Qwen3-14B-FP8RTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    202.8 tok/s
    Peak prefill
    3,018 tok/s
    by @Sandberg
  • Ministral-3-3B-Instruct-2512-FP8RTX 3060 12GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    696.2 tok/s
    Peak prefill
    1,828 tok/s
    by @Sandberg
  • Fara1.5-27B-AWQ-W4A16RTX 3090 24GB · vllm 0.32.0 · AWQ W4A164 contexts × 8 concurrency levels
    Peak decode
    332.3 tok/s
    Peak prefill
    2,144 tok/s
    by @Sandberg
  • Cosmos-Reason2-8BRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    644.4 tok/s
    Peak prefill
    3,932 tok/s
    by @Sandberg
  • Qwen3.5-4B-FP8-dynamicRTX 3060 12GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    545.9 tok/s
    Peak prefill
    2,186 tok/s
    by @Sandberg
  • Llama-3.2-3B-InstructRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    1,232.0 tok/s
    Peak prefill
    14.7K tok/s
    by @Sandberg
  • Nemotron-Cascade-2-30B-A3B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,075.9 tok/s
    Peak prefill
    8,169 tok/s
    by @Sandberg
  • Qwen3-0.6BRTX 3090 24GB · vllm 0.32.0 · BF163 contexts × 8 concurrency levels
    Peak decode
    1,756.7 tok/s
    Peak prefill
    39.4K tok/s
    by @Sandberg
  • Qwen3-1.7BRTX 3090 24GB · vllm 0.32.0 · BF163 contexts × 8 concurrency levels
    Peak decode
    1,509.4 tok/s
    Peak prefill
    23.8K tok/s
    by @Sandberg
  • Qwen3.5-4B-FP8-dynamicRTX 3060 12GB · vllm 0.32.0 · FP8 dynamic6 contexts × 8 concurrency levels
    Peak decode
    909.3 tok/s
    Peak prefill
    2,370 tok/s
    by @Sandberg
  • Ministral-3-3B-Instruct-2512RTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    2,983.4 tok/s
    Peak prefill
    12.5K tok/s
    by @Sandberg
  • Fara1.5-4BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,834.8 tok/s
    Peak prefill
    11.8K tok/s
    by @Sandberg
  • Qwen3.5-27B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit4 contexts × 8 concurrency levels
    Peak decode
    173.3 tok/s
    Peak prefill
    1,555 tok/s
    by @Sandberg
  • Qwen3-Next-80B-A3B-Instruct-FP8CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · FP86 contexts × 8 concurrency levels
    Peak decode
    2,821.5 tok/s
    Peak prefill
    10.8K tok/s
    by @Sandberg
  • Agents-A1-4B-FP8RTX 3060 12GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    890.6 tok/s
    Peak prefill
    2,341 tok/s
    by @Sandberg
  • Agents-A1-4BRTX 3090 24GB · vllm 0.32.0 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,811.6 tok/s
    Peak prefill
    11.9K tok/s
    by @Sandberg