Premiers pasClassementDecode calculatorModèlesOrganizationsReportsMatrix runsMatérielÉvaluations comparativesMarketplaceLocationsProDocs API
Langue
Actual Computer — Every computer, one endpoint

Matrix runs

A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Run one with lmx speed-test matrix run.

Clear

Matching runs 699

Submitted by @keennay. Newest first. Page 27 of 28.
  • Qwen3.5-4B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,252.4 tok/s
    Peak prefill
    43.1K tok/s
  • Qwen3.5-4B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    6,315.6 tok/s
    Peak prefill
    44.3K tok/s
  • Qwen3.5-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,747.4 tok/s
    Peak prefill
    36.3K tok/s
  • Qwen3.5-35B-A3BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,598.7 tok/s
    Peak prefill
    37K tok/s
  • Qwen3.5-35B-A3B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post2 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,629.2 tok/s
    Peak prefill
    27.6K tok/s
  • Qwen3.5-35B-A3B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post2 · INT46 contexts × 8 concurrency levels
    Peak decode
    6,237.8 tok/s
    Peak prefill
    31.7K tok/s
  • Qwen3.5-35B-A3B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    6,559.1 tok/s
    Peak prefill
    39.9K tok/s
  • Qwen3.5-35B-A3B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    6,344.8 tok/s
    Peak prefill
    40.4K tok/s
  • Qwen3.5-35B-A3B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    2,797.8 tok/s
    Peak prefill
    35.9K tok/s
  • Qwen3.5-35B-A3B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    3,351.2 tok/s
    Peak prefill
    36.5K tok/s
  • Qwen3.5-2BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    12.1K tok/s
    Peak prefill
    77.3K tok/s
  • Qwen3.5-2BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    12.3K tok/s
    Peak prefill
    80.6K tok/s
  • Qwen3.5-2B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    5,110.7 tok/s
    Peak prefill
    77.5K tok/s
  • Qwen3.5-2B-BaseH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    12.4K tok/s
    Peak prefill
    79.8K tok/s
  • Qwen3.5-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,843.2 tok/s
    Peak prefill
    9,631 tok/s
  • Qwen3.5-27BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    1,183.1 tok/s
    Peak prefill
    9,731 tok/s
  • Qwen3.5-27B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,910.6 tok/s
    Peak prefill
    6,160 tok/s
  • Qwen3.5-27B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · INT46 contexts × 8 concurrency levels
    Peak decode
    1,223.8 tok/s
    Peak prefill
    6,236 tok/s
  • Qwen3.5-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    2,263.4 tok/s
    Peak prefill
    13.3K tok/s
  • Qwen3.5-27B-FP8H200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    1,582.8 tok/s
    Peak prefill
    13.7K tok/s
  • Qwen3.5-122B-A10B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post2 · INT46 contexts × 8 concurrency levels
    Peak decode
    571.4 tok/s
    Peak prefill
    12.8K tok/s
  • Qwen3.5-122B-A10B-GPTQ-Int4H200 NVL 141GB · sglang 0.5.6.post2 · INT46 contexts × 8 concurrency levels
    Peak decode
    2,006.6 tok/s
    Peak prefill
    13.2K tok/s
  • Qwen3.5-122B-A10B-FP8H200 NVL 141GB ×2 · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · FP86 contexts × 8 concurrency levels
    Peak decode
    4,794.2 tok/s
    Peak prefill
    28.4K tok/s
  • Qwen3.5-0.8BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    12.3K tok/s
    Peak prefill
    99K tok/s
  • Qwen3.5-0.8BH200 NVL 141GB · sglang 0.5.6.post3.dev9160+g1cf2b8c54 · BF166 contexts × 8 concurrency levels
    Peak decode
    16K tok/s
    Peak prefill
    98.7K tok/s