Verified 1w ago Mainly llama.cpp 200 approved runs
Submissions
1,164
200 approved public runs
Models
28
unique models
Avg output
303tok/s
Best 3,216 tok/s
Avg prefill
330tok/s
Best 3,953 tok/s
Avg total
626tok/s
Best 6,953 tok/s
Avg TTFT
5,538ms
Best 77.2 ms
Hardware
2
distinct setups
Matrix runs
36
speed test matrices
Matrix runs36+
Matrix runs 36
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 564.5 tok/s is the aggregate decode rate across all 96 streams, about 5.9 tok/s per stream, and the 9608 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 564.3, 564.5, 564.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 566.8 tok/s is the aggregate decode rate across all 48 streams, about 11.8 tok/s per stream, and the 5550 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 566.8, 570.4, 565.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 442.6 tok/s is the aggregate decode rate across all 24 streams, about 18.4 tok/s per stream, and the 2881 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 442.6, 442.5, 443.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 81.8 tok/s is the aggregate decode rate across all 6 streams, about 13.6 tok/s per stream, and the 2345 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 88.5, 81.8, 84.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 44.8 tok/s is the aggregate decode rate across all 3 streams, about 14.9 tok/s per stream, and the 1207 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 44.8, 46.1, 44.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1044.7 tok/s is the aggregate decode rate across all 24 streams, about 43.5 tok/s per stream, and the 638 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 64, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1060.6, 1044.7, 1062.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq36-35b-fp8-ns64, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 178.1 tok/s is the aggregate decode rate across all 6 streams, about 29.7 tok/s per stream, and the 1204 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 177.8, 177.9, 178.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 97 tok/s is the aggregate decode rate across all 3 streams, about 32.3 tok/s per stream, and the 627 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 97.3, 97, 96.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 293.2 tok/s is the aggregate decode rate across all 12 streams, about 24.4 tok/s per stream, and the 1648 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 292.8, 293.2, 293.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 28.4 tok/s is the aggregate decode rate across all 16 streams, about 1.8 tok/s per stream, and the 17421 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47206 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 29.6, 28.4, 27.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 80.1 tok/s is the aggregate decode rate across all 4 streams, about 20.0 tok/s per stream, and the 2059 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF @ 8cd082170ed313484be1cf44f256a5caa2cfdad3, file NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 80.1, 79.2, 81.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 64 tok/s is the aggregate decode rate across all 2 streams, about 32.0 tok/s per stream, and the 1321 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF @ 8cd082170ed313484be1cf44f256a5caa2cfdad3, file NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 61.9, 64.2, 64 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF @ 8cd082170ed313484be1cf44f256a5caa2cfdad3, file NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 46.4, 46.9, 47.4 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 4.8 tok/s is the aggregate decode rate across all 4 streams, about 1.2 tok/s per stream, and the 7339 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.1, 4.4, 4.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 17.8 tok/s is the aggregate decode rate across all 12 streams, about 1.5 tok/s per stream, and the 12898 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47206 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 17.8, 27.1, 18.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 1.9 tok/s is the aggregate decode rate across all 2 streams, about 0.9 tok/s per stream, and the 5892 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 2.4, 1.9, 2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 86.4 tok/s is the aggregate decode rate across all 6 streams, about 14.4 tok/s per stream, and the 2503 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (317 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 86.4, 86.2, 83.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq72-awq-eager, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 40.5 tok/s is the aggregate decode rate across all 3 streams, about 13.5 tok/s per stream, and the 1278 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (319 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 38.7, 42, 40.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq72-awq-eager, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 161.8 tok/s is the aggregate decode rate across all 24 streams, about 6.7 tok/s per stream, and the 6530 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (316 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 168, 161.8, 159.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq72-awq-eager, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 5.3 tok/s is the aggregate decode rate across all 8 streams, about 0.7 tok/s per stream, and the 12889 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47142 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6, 5.1, 5.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 124.6 tok/s is the aggregate decode rate across all 12 streams, about 10.4 tok/s per stream, and the 3810 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (318 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 132.8, 129, 124.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq72-awq-eager, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 1900.6 tok/s is the aggregate decode rate across all 96 streams, about 19.8 tok/s per stream, and the 1823 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 1894.1, 1900.6, 1915 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 459.8 tok/s is the aggregate decode rate across all 6 streams, about 76.6 tok/s per stream, and the 196 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 481.6, 459.8, 475.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 16 tok/s is the aggregate decode rate across all 4 streams, about 4.0 tok/s per stream, and the 4017 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47142 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 17.3, 16.3, 16 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47142 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 26.8, 27.3, 26.1 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 564.5 tok/s is the aggregate decode rate across all 96 streams, about 5.9 tok/s per stream, and the 9608 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 564.3, 564.5, 564.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 566.8 tok/s is the aggregate decode rate across all 48 streams, about 11.8 tok/s per stream, and the 5550 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 566.8, 570.4, 565.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 442.6 tok/s is the aggregate decode rate across all 24 streams, about 18.4 tok/s per stream, and the 2881 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 442.6, 442.5, 443.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 81.8 tok/s is the aggregate decode rate across all 6 streams, about 13.6 tok/s per stream, and the 2345 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (305 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 88.5, 81.8, 84.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 44.8 tok/s is the aggregate decode rate across all 3 streams, about 14.9 tok/s per stream, and the 1207 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 44.8, 46.1, 44.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl31-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1044.7 tok/s is the aggregate decode rate across all 24 streams, about 43.5 tok/s per stream, and the 638 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 64, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1060.6, 1044.7, 1062.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq36-35b-fp8-ns64, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 178.1 tok/s is the aggregate decode rate across all 6 streams, about 29.7 tok/s per stream, and the 1204 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 177.8, 177.9, 178.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 97 tok/s is the aggregate decode rate across all 3 streams, about 32.3 tok/s per stream, and the 627 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 97.3, 97, 96.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 293.2 tok/s is the aggregate decode rate across all 12 streams, about 24.4 tok/s per stream, and the 1648 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 292.8, 293.2, 293.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-32b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 28.4 tok/s is the aggregate decode rate across all 16 streams, about 1.8 tok/s per stream, and the 17421 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47206 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 29.6, 28.4, 27.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 80.1 tok/s is the aggregate decode rate across all 4 streams, about 20.0 tok/s per stream, and the 2059 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF @ 8cd082170ed313484be1cf44f256a5caa2cfdad3, file NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 80.1, 79.2, 81.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 64 tok/s is the aggregate decode rate across all 2 streams, about 32.0 tok/s per stream, and the 1321 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF @ 8cd082170ed313484be1cf44f256a5caa2cfdad3, file NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 61.9, 64.2, 64 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF @ 8cd082170ed313484be1cf44f256a5caa2cfdad3, file NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Q4_0.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 46.4, 46.9, 47.4 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 4.8 tok/s is the aggregate decode rate across all 4 streams, about 1.2 tok/s per stream, and the 7339 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.1, 4.4, 4.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 17.8 tok/s is the aggregate decode rate across all 12 streams, about 1.5 tok/s per stream, and the 12898 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47206 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 17.8, 27.1, 18.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 1.9 tok/s is the aggregate decode rate across all 2 streams, about 0.9 tok/s per stream, and the 5892 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 78 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.8-Flash-Next-GGUF @ 766911a6b7369840a91dbcd95f9f997acaab6cd6, file Qwen3.8-Flash-Next-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 2.4, 1.9, 2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 86.4 tok/s is the aggregate decode rate across all 6 streams, about 14.4 tok/s per stream, and the 2503 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (317 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 86.4, 86.2, 83.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq72-awq-eager, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 40.5 tok/s is the aggregate decode rate across all 3 streams, about 13.5 tok/s per stream, and the 1278 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (319 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 38.7, 42, 40.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq72-awq-eager, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 161.8 tok/s is the aggregate decode rate across all 24 streams, about 6.7 tok/s per stream, and the 6530 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (316 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 168, 161.8, 159.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq72-awq-eager, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 5.3 tok/s is the aggregate decode rate across all 8 streams, about 0.7 tok/s per stream, and the 12889 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47142 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6, 5.1, 5.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 124.6 tok/s is the aggregate decode rate across all 12 streams, about 10.4 tok/s per stream, and the 3810 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (318 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 132.8, 129, 124.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq72-awq-eager, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 1900.6 tok/s is the aggregate decode rate across all 96 streams, about 19.8 tok/s per stream, and the 1823 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 1894.1, 1900.6, 1915 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 459.8 tok/s is the aggregate decode rate across all 6 streams, about 76.6 tok/s per stream, and the 196 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 481.6, 459.8, 475.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3c-30b-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 16 tok/s is the aggregate decode rate across all 4 streams, about 4.0 tok/s per stream, and the 4017 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47142 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 17.3, 16.3, 16 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 71 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47142 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF @ 99144d1d7a318f045f1412a85bbb807f3c8a17a0, file NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Q4_0.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 26.8, 27.3, 26.1 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
