Verified 1w ago Mainly llama.cpp 200 approved runs
Submissions
1,149
200 approved public runs
Models
28
unique models
Avg output
286tok/s
Best 3,216 tok/s
Avg prefill
333tok/s
Best 3,953 tok/s
Avg total
587tok/s
Best 6,953 tok/s
Avg TTFT
5,470ms
Best 77.2 ms
Hardware
2
distinct setups
Matrix runs
36
speed test matrices
Matrix runs36+
Matrix runs 36
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 1442.3 tok/s is the aggregate decode rate across all 48 streams, about 30.0 tok/s per stream, and the 961 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 1447.6, 1452, 1442.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 282.6 tok/s is the aggregate decode rate across all 3 streams, about 94.2 tok/s per stream, and the 143 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 284.6, 282.6, 270.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1012.7 tok/s is the aggregate decode rate across all 24 streams, about 42.2 tok/s per stream, and the 502 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1090.6, 1062, 1012.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 706.4 tok/s is the aggregate decode rate across all 12 streams, about 58.9 tok/s per stream, and the 291 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 706.4, 711.1, 697.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 7.5 tok/s is the aggregate decode rate across all 8 streams, about 0.9 tok/s per stream, and the 10404 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.2, 7.5, 13.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 3215.7 tok/s is the aggregate decode rate across all 96 streams, about 33.5 tok/s per stream, and the 1794 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 3251.2, 3215.7, 3186.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 622 tok/s is the aggregate decode rate across all 6 streams, about 103.7 tok/s per stream, and the 198 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 622, 599.7, 546 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 2343.2 tok/s is the aggregate decode rate across all 48 streams, about 48.8 tok/s per stream, and the 1171 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 2343.2, 2296.3, 2332.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 316.6 tok/s is the aggregate decode rate across all 3 streams, about 105.5 tok/s per stream, and the 131 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (333 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 316.6, 314.4, 388.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1650.1 tok/s is the aggregate decode rate across all 24 streams, about 68.8 tok/s per stream, and the 624 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (335 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1573.6, 1650.1, 1705.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 76.6 tok/s is the aggregate decode rate across all 6 streams, about 12.8 tok/s per stream, and the 2348 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (306 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 71.2, 76.6, 79.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 39 tok/s is the aggregate decode rate across all 3 streams, about 13.0 tok/s per stream, and the 1203 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 38.4, 39.3, 39 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 14.4 tok/s is the aggregate decode rate across all 4 streams, about 3.6 tok/s per stream, and the 7547 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 13.2, 11.5, 14.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 11.9 tok/s is the aggregate decode rate across all 2 streams, about 6.0 tok/s per stream, and the 3645 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.2, 11.9, 11.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 27, 25.2, 28 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 222.3 tok/s is the aggregate decode rate across all 24 streams, about 9.3 tok/s per stream, and the 6360 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 217, 230.2, 222.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 144.4 tok/s is the aggregate decode rate across all 12 streams, about 12.0 tok/s per stream, and the 3373 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 144.4, 125.9, 151.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 872.4 tok/s is the aggregate decode rate across all 12 streams, about 72.7 tok/s per stream, and the 341 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (335 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 950.9, 872.4, 935.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 360 tok/s is the aggregate decode rate across all 32 streams, about 11.2 tok/s per stream, and the 6225 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47406 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 360, 361.5, 341.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 183.1 tok/s is the aggregate decode rate across all 24 streams, about 7.6 tok/s per stream, and the 4396 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47406 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 182.4, 243, 183.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 116.4 tok/s is the aggregate decode rate across all 32 streams, about 3.6 tok/s per stream, and the 43540 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 60 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47466 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 111.1, 118.7, 116.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
--d
--name lsv-ornith122-q2kxl-np8
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 170.1 tok/s is the aggregate decode rate across all 8 streams, about 21.3 tok/s per stream, and the 2529 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 170.1, 179.2, 182.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
--d
--name lsv-ornith122-q2kxl-np8
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 156.7 tok/s is the aggregate decode rate across all 4 streams, about 39.2 tok/s per stream, and the 1343 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 158.5, 157.5, 156.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
--d
--name lsv-ornith122-q2kxl-np8
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 117.4 tok/s is the aggregate decode rate across all 2 streams, about 58.7 tok/s per stream, and the 810 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 117.9, 117.4, 116.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
--d
--name lsv-ornith122-q2kxl-np8
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 72.7, 72, 71.4 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 1442.3 tok/s is the aggregate decode rate across all 48 streams, about 30.0 tok/s per stream, and the 961 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 1447.6, 1452, 1442.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 282.6 tok/s is the aggregate decode rate across all 3 streams, about 94.2 tok/s per stream, and the 143 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (295 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 284.6, 282.6, 270.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1012.7 tok/s is the aggregate decode rate across all 24 streams, about 42.2 tok/s per stream, and the 502 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (296 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1090.6, 1062, 1012.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 706.4 tok/s is the aggregate decode rate across all 12 streams, about 58.9 tok/s per stream, and the 291 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (297 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 706.4, 711.1, 697.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vq3-30b-2507-fp8, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 7.5 tok/s is the aggregate decode rate across all 8 streams, about 0.9 tok/s per stream, and the 10404 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.2, 7.5, 13.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 96 requests sent at once (batch 96, lmx --concurrency 96); 3215.7 tok/s is the aggregate decode rate across all 96 streams, about 33.5 tok/s per stream, and the 1794 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 96: 3251.2, 3215.7, 3186.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 622 tok/s is the aggregate decode rate across all 6 streams, about 103.7 tok/s per stream, and the 198 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 622, 599.7, 546 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 48 requests sent at once (batch 48, lmx --concurrency 48); 2343.2 tok/s is the aggregate decode rate across all 48 streams, about 48.8 tok/s per stream, and the 1171 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (334 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 48: 2343.2, 2296.3, 2332.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 316.6 tok/s is the aggregate decode rate across all 3 streams, about 105.5 tok/s per stream, and the 131 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (333 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 316.6, 314.4, 388.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 1650.1 tok/s is the aggregate decode rate across all 24 streams, about 68.8 tok/s per stream, and the 624 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (335 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 1573.6, 1650.1, 1705.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (batch 6, lmx --concurrency 6); 76.6 tok/s is the aggregate decode rate across all 6 streams, about 12.8 tok/s per stream, and the 2348 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (306 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 6: 71.2, 76.6, 79.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (batch 3, lmx --concurrency 3); 39 tok/s is the aggregate decode rate across all 3 streams, about 13.0 tok/s per stream, and the 1203 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (304 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 3: 38.4, 39.3, 39 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 14.4 tok/s is the aggregate decode rate across all 4 streams, about 3.6 tok/s per stream, and the 7547 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 13.2, 11.5, 14.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 11.9 tok/s is the aggregate decode rate across all 2 streams, about 6.0 tok/s per stream, and the 3645 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.2, 11.9, 11.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47748 MiB of VRAM in use after load; a model download was running on the VM during this run, 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 27, 25.2, 28 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (batch 24, lmx --concurrency 24); 222.3 tok/s is the aggregate decode rate across all 24 streams, about 9.3 tok/s per stream, and the 6360 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 24: 217, 230.2, 222.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 144.4 tok/s is the aggregate decode rate across all 12 streams, about 12.0 tok/s per stream, and the 3373 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs 32, max_model_len 4096; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (303 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 144.4, 125.9, 151.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config vl33-70b-awq, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (batch 12, lmx --concurrency 12); 872.4 tok/s is the aggregate decode rate across all 12 streams, about 72.7 tok/s per stream, and the 341 ms time to first token is under that load. Setup: KASA cuda-vm, 1 x NVIDIA RTX A6000 48 GB passed through to a VM (28 vCPU of an AMD EPYC 7343, 128 GiB), one card; vLLM 0.32.0 (vllm/vllm-openai@sha256:befccf0e7e4089aa23926bb6f887a85c0d46c2a99e6de5c5ffdf5cb7957c5b34); max_num_seqs ?, max_model_len 32768; speculative decoding off. Each request: LocalMaxxing's canonical reasoning-v1 prompt (335 tokens) with a unique nonce, 256 output tokens, temperature 0. One discarded warm-up batch, then 3 measured batches of 12: 950.9, 872.4, 935.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Config voss20-mxfp4, kasa-testing campaign ai/vm/cuda-vm/2026-10-26-a6000-localmaxxing. Selection declared before measuring: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional mean, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 360 tok/s is the aggregate decode rate across all 32 streams, about 11.2 tok/s per stream, and the 6225 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47406 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 360, 361.5, 341.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 24 requests sent at once (lmx --concurrency 24); 183.1 tok/s is the aggregate decode rate across all 24 streams, about 7.6 tok/s per stream, and the 4396 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47406 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 182.4, 243, 183.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 32 requests sent at once (lmx --concurrency 32); 116.4 tok/s is the aggregate decode rate across all 32 streams, about 3.6 tok/s per stream, and the 43540 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 60 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47466 MiB of VRAM in use after load, 16 generation threads, 28 batch threads, flash attention on, 32 server slots of 4096 tokens each, context 131072. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 111.1, 118.7, 116.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
--d
--name lsv-ornith122-q2kxl-np8
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 170.1 tok/s is the aggregate decode rate across all 8 streams, about 21.3 tok/s per stream, and the 2529 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 170.1, 179.2, 182.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
--d
--name lsv-ornith122-q2kxl-np8
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 156.7 tok/s is the aggregate decode rate across all 4 streams, about 39.2 tok/s per stream, and the 1343 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 158.5, 157.5, 156.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
--d
--name lsv-ornith122-q2kxl-np8
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 117.4 tok/s is the aggregate decode rate across all 2 streams, about 58.7 tok/s per stream, and the 810 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 117.9, 117.4, 116.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
--d
--name lsv-ornith122-q2kxl-np8
--gpus all
--ipc host
--p 127.0.0.1:8000:8000
--e HF_HOME=/mnt/models/hf
--jinja
--tb 28
--metrics
--fit off
--b 4096
Notes
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 28 vCPU of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), all layers on the GPU (-ngl 99), 16 generation threads, 28 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Ornith-1.5-122B-GGUF @ 745bd742473a6cdddb1c6f78fb9a3b3a9293bd41, file Ornith-1.5-122B-UD-Q2_K_XL.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 72.7, 72, 71.4 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
