Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 351

Submitted by @Sandberg. Newest first. Page 1 of 15.
  • TwIL-8BRTX 3090 24GB · vllm 0.32.0 · BF165 contexts × 8 concurrency levels
    Peak decode
    311.1 tok/s
    Peak prefill
    3,234 tok/s
  • TwIL-8BCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · BF165 contexts × 8 concurrency levels
    Peak decode
    1,393.6 tok/s
    Peak prefill
    6,607 tok/s
  • Apex-35B-V1.0CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · BF166 contexts × 8 concurrency levels
    Peak decode
    2,901.5 tok/s
    Peak prefill
    11.7K tok/s
  • Carbon-XRTX 3090 24GB · vllm 0.32.0 · FP83 contexts × 8 concurrency levels
    Peak decode
    200.7 tok/s
    Peak prefill
    3,110 tok/s
  • Qwen3.5-122B-A10B-AWQ-4bitCMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · AWQ 4-bit6 contexts × 8 concurrency levels
    Peak decode
    1,613.3 tok/s
    Peak prefill
    5,659 tok/s
  • Qwen3.5-122B-A10B-NVFP4CMP 170HX 64GB ×2 · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    1,066.4 tok/s
    Peak prefill
    4,420 tok/s
  • Qwen3.5-9B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    1,687.2 tok/s
    Peak prefill
    5,941 tok/s
  • Qwen3.5-9B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    1,788.6 tok/s
    Peak prefill
    6,692 tok/s
  • Qwen3.5-9B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · W4A16 g323 contexts × 8 concurrency levels
    Peak decode
    345.3 tok/s
    Peak prefill
    1,593 tok/s
  • Qwen3.5-9B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    3,107.6 tok/s
    Peak prefill
    5,489 tok/s
  • Qwen3.5-9B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    3,300.3 tok/s
    Peak prefill
    6,111 tok/s
  • Qwen3.5-9B-AWQ-4bitRTX 3060 12GB · vllm 0.32.0 · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    775.5 tok/s
    Peak prefill
    1,473 tok/s
  • Qwen3.5-9B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    2,087.6 tok/s
    Peak prefill
    6,266 tok/s
  • Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTPCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    807.4 tok/s
    Peak prefill
    1,910 tok/s
  • Qwen3.5-9B-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    2,140.9 tok/s
    Peak prefill
    7,108 tok/s
  • Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTPCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    791.5 tok/s
    Peak prefill
    1,944 tok/s
  • Muse-Glimmer-30B-NVFP4-W4A4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP4 W4A45 contexts × 8 concurrency levels
    Peak decode
    1,320.9 tok/s
    Peak prefill
    1,954 tok/s
  • Thomson-1.0-Small-NVFP4CMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · NVFP46 contexts × 8 concurrency levels
    Peak decode
    3,224.7 tok/s
    Peak prefill
    7,178 tok/s
  • Qwen3.6-27B-AWQCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ W4 g1286 contexts × 8 concurrency levels
    Peak decode
    964.3 tok/s
    Peak prefill
    1,921 tok/s
  • Qwen3.6-27B-AWQCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · AWQ W4 g1286 contexts × 8 concurrency levels
    Peak decode
    1,161.1 tok/s
    Peak prefill
    1,959 tok/s
  • Qwen3.5-9B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    2,713.1 tok/s
    Peak prefill
    5,983 tok/s
  • Qwen3.5-9B-AWQ-4bitCMP 170HX 64GB · vllm 0.32.0rc1.dev249+g18305948a · W4A16 g326 contexts × 8 concurrency levels
    Peak decode
    3,715.5 tok/s
    Peak prefill
    6,231 tok/s
  • Hy-MT2-7B-FP8RTX 3090 24GB · vllm 0.32.0 · FP86 contexts × 8 concurrency levels
    Peak decode
    632.9 tok/s
    Peak prefill
    3,997 tok/s
  • Tess-4-27B-NVFP4RTX 3090 24GB · vllm 0.32.0 · NVFP43 contexts × 8 concurrency levels
    Peak decode
    160.0 tok/s
    Peak prefill
    1,823 tok/s
  • Qwen3-Coder-30B-A3B-Instruct-AWQ-4bitRTX 3090 24GB · vllm 0.32.0 · AWQ 4-bit4 contexts × 8 concurrency levels
    Peak decode
    753.1 tok/s
    Peak prefill
    8,583 tok/s