Verified 1w ago Mainly llama.cpp 200 approved runs
Submissions
1,114
200 approved public runs
Models
24
unique models
Avg output
367tok/s
Best 3,216 tok/s
Avg prefill
376tok/s
Best 3,953 tok/s
Avg total
763tok/s
Best 6,953 tok/s
Avg TTFT
5,093ms
Best 77.2 ms
Hardware
2
distinct setups
Matrix runs
36
speed test matrices
Matrix runs36+
Matrix runs 36
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 1569.6 tok/s is the aggregate decode rate across all 48 streams, about 32.7 tok/s per stream, and the 988 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 1569.6, 1572, 1555.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 8.7 tok/s is the aggregate decode rate across all 12 streams, about 0.7 tok/s per stream, and the 11488 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47518 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.7, 13.5, 6.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: VRAM_RESERVE_MIB=1024. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 38.2, 40.7, 35.4 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.8. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 32.6, 40.7, 48.3 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 37.2, 44, 50.2 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: PCIE_FRAC=0.9. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 27.8, 41.3, 38.1 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: PCIE_FRAC=0.5. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 34.6, 34.4, 29.7 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 30 tok/s is the aggregate decode rate across all 4 streams, about 7.5 tok/s per stream. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: recipe defaults. Canonical reasoning-v1 prompt with a unique nonce; one discarded warm-up batch, then 3 measured batches: 29.4, 30, 30.9 tok/s; this is the median. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 37.1 tok/s is the aggregate decode rate across all 2 streams, about 18.6 tok/s per stream. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: recipe defaults. Canonical reasoning-v1 prompt with a unique nonce; one discarded warm-up batch, then 3 measured batches: 40.1, 43.4, 37.1 tok/s; this is the median. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: recipe defaults. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 34.3, 40.4, 43.2 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 275.2 tok/s is the aggregate decode rate across all 3 streams, about 91.7 tok/s per stream, and the 143 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 268.1, 292.9, 275.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1123.7 tok/s is the aggregate decode rate across all 24 streams, about 46.8 tok/s per stream, and the 515 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1099.4, 1123.7, 1138.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 692.9 tok/s is the aggregate decode rate across all 12 streams, about 57.7 tok/s per stream, and the 286 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 687.4, 720.1, 692.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 1820.2 tok/s is the aggregate decode rate across all 96 streams, about 19.0 tok/s per stream, and the 1769 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 1820.2, 1811.3, 1827.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 450.5 tok/s is the aggregate decode rate across all 6 streams, about 75.1 tok/s per stream, and the 232 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 486.8, 467.1, 450.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 1442.3 tok/s is the aggregate decode rate across all 48 streams, about 30.0 tok/s per stream, and the 961 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 1447.6, 1452, 1442.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 282.6 tok/s is the aggregate decode rate across all 3 streams, about 94.2 tok/s per stream, and the 143 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 284.6, 282.6, 270.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1012.7 tok/s is the aggregate decode rate across all 24 streams, about 42.2 tok/s per stream, and the 502 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1090.6, 1062, 1012.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 706.4 tok/s is the aggregate decode rate across all 12 streams, about 58.9 tok/s per stream, and the 291 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 706.4, 711.1, 697.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 7.5 tok/s is the aggregate decode rate across all 8 streams, about 0.9 tok/s per stream, and the 10404 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.2, 7.5, 13.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 3215.7 tok/s is the aggregate decode rate across all 96 streams, about 33.5 tok/s per stream, and the 1794 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 3251.2, 3215.7, 3186.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 622 tok/s is the aggregate decode rate across all 6 streams, about 103.7 tok/s per stream, and the 198 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 622, 599.7, 546 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 2343.2 tok/s is the aggregate decode rate across all 48 streams, about 48.8 tok/s per stream, and the 1171 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 2343.2, 2296.3, 2332.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 316.6 tok/s is the aggregate decode rate across all 3 streams, about 105.5 tok/s per stream, and the 131 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (333 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 316.6, 314.4, 388.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1650.1 tok/s is the aggregate decode rate across all 24 streams, about 68.8 tok/s per stream, and the 624 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (335 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1573.6, 1650.1, 1705.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 1569.6 tok/s is the aggregate decode rate across all 48 streams, about 32.7 tok/s per stream, and the 988 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 1569.6, 1572, 1555.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 8.7 tok/s is the aggregate decode rate across all 12 streams, about 0.7 tok/s per stream, and the 11488 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47518 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.7, 13.5, 6.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: VRAM_RESERVE_MIB=1024. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 38.2, 40.7, 35.4 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.8. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 32.6, 40.7, 48.3 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 37.2, 44, 50.2 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: PCIE_FRAC=0.9. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 27.8, 41.3, 38.1 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: PCIE_FRAC=0.5. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 34.6, 34.4, 29.7 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 30 tok/s is the aggregate decode rate across all 4 streams, about 7.5 tok/s per stream. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: recipe defaults. Canonical reasoning-v1 prompt with a unique nonce; one discarded warm-up batch, then 3 measured batches: 29.4, 30, 30.9 tok/s; this is the median. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 37.1 tok/s is the aggregate decode rate across all 2 streams, about 18.6 tok/s per stream. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: recipe defaults. Canonical reasoning-v1 prompt with a unique nonce; one discarded warm-up batch, then 3 measured batches: 40.1, 43.4, 37.1 tok/s; this is the median. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: recipe defaults. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 34.3, 40.4, 43.2 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 275.2 tok/s is the aggregate decode rate across all 3 streams, about 91.7 tok/s per stream, and the 143 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 268.1, 292.9, 275.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1123.7 tok/s is the aggregate decode rate across all 24 streams, about 46.8 tok/s per stream, and the 515 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1099.4, 1123.7, 1138.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 692.9 tok/s is the aggregate decode rate across all 12 streams, about 57.7 tok/s per stream, and the 286 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 687.4, 720.1, 692.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 1820.2 tok/s is the aggregate decode rate across all 96 streams, about 19.0 tok/s per stream, and the 1769 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 1820.2, 1811.3, 1827.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 450.5 tok/s is the aggregate decode rate across all 6 streams, about 75.1 tok/s per stream, and the 232 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 486.8, 467.1, 450.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 1442.3 tok/s is the aggregate decode rate across all 48 streams, about 30.0 tok/s per stream, and the 961 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 1447.6, 1452, 1442.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 282.6 tok/s is the aggregate decode rate across all 3 streams, about 94.2 tok/s per stream, and the 143 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 284.6, 282.6, 270.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1012.7 tok/s is the aggregate decode rate across all 24 streams, about 42.2 tok/s per stream, and the 502 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1090.6, 1062, 1012.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 706.4 tok/s is the aggregate decode rate across all 12 streams, about 58.9 tok/s per stream, and the 291 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 706.4, 711.1, 697.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 7.5 tok/s is the aggregate decode rate across all 8 streams, about 0.9 tok/s per stream, and the 10404 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.2, 7.5, 13.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 3215.7 tok/s is the aggregate decode rate across all 96 streams, about 33.5 tok/s per stream, and the 1794 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 3251.2, 3215.7, 3186.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 622 tok/s is the aggregate decode rate across all 6 streams, about 103.7 tok/s per stream, and the 198 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 622, 599.7, 546 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 2343.2 tok/s is the aggregate decode rate across all 48 streams, about 48.8 tok/s per stream, and the 1171 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 2343.2, 2296.3, 2332.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 316.6 tok/s is the aggregate decode rate across all 3 streams, about 105.5 tok/s per stream, and the 131 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (333 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 316.6, 314.4, 388.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1650.1 tok/s is the aggregate decode rate across all 24 streams, about 68.8 tok/s per stream, and the 624 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (335 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1573.6, 1650.1, 1705.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
