Verified 1w ago Mainly llama.cpp 200 approved runs
Submissions
1,123
200 approved public runs
Models
24
unique models
Avg output
333tok/s
Best 3,216 tok/s
Avg prefill
372tok/s
Best 3,953 tok/s
Avg total
688tok/s
Best 6,953 tok/s
Avg TTFT
5,197ms
Best 77.2 ms
Hardware
2
distinct setups
Matrix runs
36
speed test matrices
Matrix runs36+
Matrix runs 36
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
--d
--name lsv-ornith122-q2kxl-np16
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 156.9 tok/s is the aggregate decode rate across all 12 streams, about 13.1 tok/s per stream, and the 4094 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 156.9, 207.6, 155.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 192 tok/s is the aggregate decode rate across all 8 streams, about 24.0 tok/s per stream, and the 1944 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 192.1, 192, 192.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-nomtp, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 364 tok/s is the aggregate decode rate across all 32 streams, about 11.4 tok/s per stream, and the 5893 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 364.2, 364, 363.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-nomtp, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Config q27i4-nomtp, vLLM 0.32.0rc1.dev100+g77154ca61. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 32.4, 32.4, 32.4 tok/s; this is the median (declared rule, tie to the earlier). Prompt/output samples, engineTimingsRaw and MTP counts were added from the lmx run file and exact per-request vLLM /metrics deltas, because lmx does not send them.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 292.9 tok/s is the aggregate decode rate across all 32 streams, about 9.2 tok/s per stream, and the 12263 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 293.4, 292.9, 292 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-mtp4, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 266.7 tok/s is the aggregate decode rate across all 16 streams, about 16.7 tok/s per stream, and the 2920 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 267.8, 267.9, 266.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-mtp4, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 61.3 tok/s is the aggregate decode rate across all 24 streams, about 2.6 tok/s per stream, and the 11737 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 60 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47466 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 61.3, 71.5, 52.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 114.8 tok/s is the aggregate decode rate across all 32 streams, about 3.6 tok/s per stream, and the 16236 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 63 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47206 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF ggml-org/gpt-oss-120b-GGUF @ 7200b74bea78b0a2964ef9f3897eefa401e890ff, file gpt-oss-120b-MXFP4.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 112.3, 114.8, 117.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 74.8 tok/s is the aggregate decode rate across all 24 streams, about 3.1 tok/s per stream, and the 17847 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 63 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47206 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF ggml-org/gpt-oss-120b-GGUF @ 7200b74bea78b0a2964ef9f3897eefa401e890ff, file gpt-oss-120b-MXFP4.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 64.5, 74.8, 92.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 11.3 tok/s is the aggregate decode rate across all 24 streams, about 0.5 tok/s per stream, and the 28698 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 67 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47692 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-old-GGUF @ 91d32823a1855f7ca12919dffa86701254271eb9, file meta-llama_Llama-4-Scout-17B-16E-Instruct-Q4_K_M-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 31.9, 43.2, 11.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 230.5 tok/s is the aggregate decode rate across all 32 streams, about 7.2 tok/s per stream, and the 59930 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47606 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 210.4, 230.5, 243.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 279.7 tok/s is the aggregate decode rate across all 8 streams, about 35.0 tok/s per stream, and the 1981 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 284.5, 277, 279.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-mtp4, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 189.9 tok/s is the aggregate decode rate across all 4 streams, about 47.5 tok/s per stream, and the 1057 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (345 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 189.9, 191.6, 196.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-mtp4, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Config q27i4-mtp4, vLLM 0.32.0rc1.dev100+g77154ca61. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 69.4, 69.4, 70.4 tok/s; this is the median (declared rule, tie to the earlier). Prompt/output samples, engineTimingsRaw and MTP counts were added from the lmx run file and exact per-request vLLM /metrics deltas, because lmx does not send them.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 137.8 tok/s is the aggregate decode rate across all 24 streams, about 5.7 tok/s per stream, and the 16964 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47606 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 156.8, 123.9, 137.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 130.3 tok/s is the aggregate decode rate across all 32 streams, about 4.1 tok/s per stream, and the 63793 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 93 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47304 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 130.3, 139.7, 134.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 49.8 tok/s is the aggregate decode rate across all 24 streams, about 2.1 tok/s per stream, and the 16685 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 93 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47304 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 64.6, 49.8, 46.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 54.7 tok/s is the aggregate decode rate across all 32 streams, about 1.7 tok/s per stream, and the 32583 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47524 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 52.9, 54.7, 59.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 32.8 tok/s is the aggregate decode rate across all 16 streams, about 2.0 tok/s per stream, and the 9500 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47426 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 32.8, 33.9, 27.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 73.8 tok/s is the aggregate decode rate across all 16 streams, about 4.6 tok/s per stream, and the 6929 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 59 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47370 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-old-GGUF @ 91d32823a1855f7ca12919dffa86701254271eb9, file meta-llama_Llama-4-Scout-17B-16E-Instruct-IQ4_XS-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 80.5, 73.8, 73.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 17.4 tok/s is the aggregate decode rate across all 16 streams, about 1.1 tok/s per stream, and the 5749 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47564 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 17.4, 155.2, 30 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 406.1 tok/s is the aggregate decode rate across all 16 streams, about 25.4 tok/s per stream, and the 2335 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 421.2, 406.1, 417.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 265 tok/s is the aggregate decode rate across all 12 streams, about 22.1 tok/s per stream, and the 1654 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 265, 388.9, 261 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 54.1 tok/s is the aggregate decode rate across all 12 streams, about 4.5 tok/s per stream, and the 4772 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 59 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47370 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-old-GGUF @ 91d32823a1855f7ca12919dffa86701254271eb9, file meta-llama_Llama-4-Scout-17B-16E-Instruct-IQ4_XS-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 54.1, 64, 46.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 27.4 tok/s is the aggregate decode rate across all 12 streams, about 2.3 tok/s per stream, and the 5118 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47564 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 106.8, 35.8, 27.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
--d
--name lsv-ornith122-q2kxl-np16
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 156.9 tok/s is the aggregate decode rate across all 12 streams, about 13.1 tok/s per stream, and the 4094 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 156.9, 207.6, 155.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 192 tok/s is the aggregate decode rate across all 8 streams, about 24.0 tok/s per stream, and the 1944 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 192.1, 192, 192.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-nomtp, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 364 tok/s is the aggregate decode rate across all 32 streams, about 11.4 tok/s per stream, and the 5893 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 364.2, 364, 363.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-nomtp, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Config q27i4-nomtp, vLLM 0.32.0rc1.dev100+g77154ca61. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 32.4, 32.4, 32.4 tok/s; this is the median (declared rule, tie to the earlier). Prompt/output samples, engineTimingsRaw and MTP counts were added from the lmx run file and exact per-request vLLM /metrics deltas, because lmx does not send them.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (batch 32, lmx --concurrency 32); 292.9 tok/s is the aggregate decode rate across all 32 streams, about 9.2 tok/s per stream, and the 12263 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 32: 293.4, 292.9, 292 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-mtp4, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (batch 16, lmx --concurrency 16); 266.7 tok/s is the aggregate decode rate across all 16 streams, about 16.7 tok/s per stream, and the 2920 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 16: 267.8, 267.9, 266.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-mtp4, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 61.3 tok/s is the aggregate decode rate across all 24 streams, about 2.6 tok/s per stream, and the 11737 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 60 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47466 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 61.3, 71.5, 52.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 114.8 tok/s is the aggregate decode rate across all 32 streams, about 3.6 tok/s per stream, and the 16236 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 63 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47206 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF ggml-org/gpt-oss-120b-GGUF @ 7200b74bea78b0a2964ef9f3897eefa401e890ff, file gpt-oss-120b-MXFP4.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 112.3, 114.8, 117.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 74.8 tok/s is the aggregate decode rate across all 24 streams, about 3.1 tok/s per stream, and the 17847 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 63 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47206 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF ggml-org/gpt-oss-120b-GGUF @ 7200b74bea78b0a2964ef9f3897eefa401e890ff, file gpt-oss-120b-MXFP4.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 64.5, 74.8, 92.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 11.3 tok/s is the aggregate decode rate across all 24 streams, about 0.5 tok/s per stream, and the 28698 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 67 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47692 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-old-GGUF @ 91d32823a1855f7ca12919dffa86701254271eb9, file meta-llama_Llama-4-Scout-17B-16E-Instruct-Q4_K_M-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 31.9, 43.2, 11.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 230.5 tok/s is the aggregate decode rate across all 32 streams, about 7.2 tok/s per stream, and the 59930 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47606 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 210.4, 230.5, 243.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (batch 8, lmx --concurrency 8); 279.7 tok/s is the aggregate decode rate across all 8 streams, about 35.0 tok/s per stream, and the 1981 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (346 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 8: 284.5, 277, 279.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-mtp4, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (batch 4, lmx --concurrency 4); 189.9 tok/s is the aggregate decode rate across all 4 streams, about 47.5 tok/s per stream, and the 1057 ms time to first token is under that load. Setup: KASA b65-vm, 1 x Intel Arc Pro B65 32 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB, Ubuntu 26.04, xe driver), one card; vLLM 0.32.0rc1.dev100+g77154ca61 (vllm/vllm-openai-xpu); max_num_seqs 32, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (345 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 4: 189.9, 191.6, 196.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config q27i4-mtp4, kasa-testing campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Config q27i4-mtp4, vLLM 0.32.0rc1.dev100+g77154ca61. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 69.4, 69.4, 70.4 tok/s; this is the median (declared rule, tie to the earlier). Prompt/output samples, engineTimingsRaw and MTP counts were added from the lmx run file and exact per-request vLLM /metrics deltas, because lmx does not send them.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 137.8 tok/s is the aggregate decode rate across all 24 streams, about 5.7 tok/s per stream, and the 16964 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47606 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 156.8, 123.9, 137.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 130.3 tok/s is the aggregate decode rate across all 32 streams, about 4.1 tok/s per stream, and the 63793 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 93 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47304 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 130.3, 139.7, 134.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 49.8 tok/s is the aggregate decode rate across all 24 streams, about 2.1 tok/s per stream, and the 16685 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 93 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47304 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 64.6, 49.8, 46.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 54.7 tok/s is the aggregate decode rate across all 32 streams, about 1.7 tok/s per stream, and the 32583 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47524 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 52.9, 54.7, 59.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 32.8 tok/s is the aggregate decode rate across all 16 streams, about 2.0 tok/s per stream, and the 9500 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47426 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 32.8, 33.9, 27.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 73.8 tok/s is the aggregate decode rate across all 16 streams, about 4.6 tok/s per stream, and the 6929 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 59 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47370 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-old-GGUF @ 91d32823a1855f7ca12919dffa86701254271eb9, file meta-llama_Llama-4-Scout-17B-16E-Instruct-IQ4_XS-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 80.5, 73.8, 73.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 17.4 tok/s is the aggregate decode rate across all 16 streams, about 1.1 tok/s per stream, and the 5749 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47564 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 17.4, 155.2, 30 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 406.1 tok/s is the aggregate decode rate across all 16 streams, about 25.4 tok/s per stream, and the 2335 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 421.2, 406.1, 417.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 265 tok/s is the aggregate decode rate across all 12 streams, about 22.1 tok/s per stream, and the 1654 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q3_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 265, 388.9, 261 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 54.1 tok/s is the aggregate decode rate across all 12 streams, about 4.5 tok/s per stream, and the 4772 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 59 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47370 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-old-GGUF @ 91d32823a1855f7ca12919dffa86701254271eb9, file meta-llama_Llama-4-Scout-17B-16E-Instruct-IQ4_XS-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 54.1, 64, 46.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 27.4 tok/s is the aggregate decode rate across all 12 streams, about 2.3 tok/s per stream, and the 5118 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47564 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 106.8, 35.8, 27.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
