Verified 3w ago Mainly llama.cpp 200 approved runs
Submissions
2,318
200 approved public runs
Models
16
unique models
Avg output
24.0tok/s
Best 103 tok/s
Avg prefill
72.7tok/s
Best 281 tok/s
Avg total
43.4tok/s
Best 215 tok/s
Avg TTFT
10,220ms
Best 1,110 ms
Hardware
2
distinct setups
Matrix runs
36
speed test matrices
Matrix runs36+
Matrix runs 36
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 67.8 tok/s is the aggregate decode rate across all 16 streams, about 4.2 tok/s per stream, and the 21442 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 60.9, 67.8, 65.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7, 7.3, 6.9 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 6.9 tok/s is the aggregate decode rate across all 6 streams, about 1.2 tok/s per stream, and the 8262 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47508 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.8, 6.9, 4.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 33.7 tok/s is the aggregate decode rate across all 8 streams, about 4.2 tok/s per stream, and the 9458 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 33.7, 35.8, 32.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 42.3 tok/s is the aggregate decode rate across all 12 streams, about 3.5 tok/s per stream, and the 15045 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 42.3, 53.3, 51.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 29.4, 29.5, 28.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 68.2 tok/s is the aggregate decode rate across all 16 streams, about 4.3 tok/s per stream, and the 9269 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 38 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-IQ4_XS.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 82.2, 68.2, 76.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 6.6 tok/s is the aggregate decode rate across all 6 streams, about 1.1 tok/s per stream, and the 8511 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47318 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.7, 5.3, 6.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 38.3 tok/s is the aggregate decode rate across all 8 streams, about 4.8 tok/s per stream, and the 7057 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 38 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-IQ4_XS.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 40.8, 59.7, 38.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 47.1 tok/s is the aggregate decode rate across all 12 streams, about 3.9 tok/s per stream, and the 7970 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 38 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-IQ4_XS.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 47.1, 78.6, 58.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 5.5 tok/s is the aggregate decode rate across all 3 streams, about 1.8 tok/s per stream, and the 10690 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47318 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 3.9, 5.5, 6.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 68.8 tok/s is the aggregate decode rate across all 16 streams, about 4.3 tok/s per stream, and the 16014 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 68.8, 75.7, 61.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 65.6 tok/s is the aggregate decode rate across all 12 streams, about 5.5 tok/s per stream, and the 10852 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 49.1, 65.6, 49.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 38 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-IQ4_XS.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 33.9, 34.3, 31.8 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 60.1 tok/s is the aggregate decode rate across all 8 streams, about 7.5 tok/s per stream, and the 7723 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 44.4, 60.1, 49.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 42.3 tok/s is the aggregate decode rate across all 4 streams, about 10.6 tok/s per stream, and the 3957 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 42.3, 43.8, 37.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 40.8, 37.9, 40.6 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 20.2 tok/s is the aggregate decode rate across all 16 streams, about 1.3 tok/s per stream, and the 33238 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 113 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/GLM-4.6V-GGUF @ 7de678a269f576ae4090efbf59b558a993386fff, file GLM-4.6V-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 20.2, 19.2, 23.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 8.6 tok/s is the aggregate decode rate across all 4 streams, about 2.1 tok/s per stream, and the 3689 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 67 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-old-GGUF @ 91d32823a1855f7ca12919dffa86701254271eb9, file meta-llama_Llama-4-Scout-17B-16E-Instruct-Q4_K_M-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.6, 11.6, 12.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 14.5 tok/s is the aggregate decode rate across all 12 streams, about 1.2 tok/s per stream, and the 27321 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 113 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/GLM-4.6V-GGUF @ 7de678a269f576ae4090efbf59b558a993386fff, file GLM-4.6V-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.5, 18.8, 14.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 16.1 tok/s is the aggregate decode rate across all 8 streams, about 2.0 tok/s per stream, and the 3976 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 63 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47190 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/gpt-oss-120b-GGUF @ 7200b74bea78b0a2964ef9f3897eefa401e890ff, file gpt-oss-120b-MXFP4.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 18.6, 16.1, 18.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 7.7 tok/s is the aggregate decode rate across all 4 streams, about 1.9 tok/s per stream, and the 4815 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 77 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47380 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.9, 7.7, 8.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 11.2 tok/s is the aggregate decode rate across all 8 streams, about 1.4 tok/s per stream, and the 23043 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 113 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/GLM-4.6V-GGUF @ 7de678a269f576ae4090efbf59b558a993386fff, file GLM-4.6V-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 11.3, 16.3, 11.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 7.5 tok/s is the aggregate decode rate across all 2 streams, about 3.8 tok/s per stream, and the 1864 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 60 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47312 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.9, 7.5, 6.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 11.7 tok/s is the aggregate decode rate across all 4 streams, about 2.9 tok/s per stream, and the 12606 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 113 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/GLM-4.6V-GGUF @ 7de678a269f576ae4090efbf59b558a993386fff, file GLM-4.6V-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 13.6, 13.1, 11.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 67.8 tok/s is the aggregate decode rate across all 16 streams, about 4.2 tok/s per stream, and the 21442 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 60.9, 67.8, 65.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7, 7.3, 6.9 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 6.9 tok/s is the aggregate decode rate across all 6 streams, about 1.2 tok/s per stream, and the 8262 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47508 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.8, 6.9, 4.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 33.7 tok/s is the aggregate decode rate across all 8 streams, about 4.2 tok/s per stream, and the 9458 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 33.7, 35.8, 32.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 42.3 tok/s is the aggregate decode rate across all 12 streams, about 3.5 tok/s per stream, and the 15045 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 42.3, 53.3, 51.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 84 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-Q8_0-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 29.4, 29.5, 28.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 68.2 tok/s is the aggregate decode rate across all 16 streams, about 4.3 tok/s per stream, and the 9269 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 38 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-IQ4_XS.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 82.2, 68.2, 76.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 6.6 tok/s is the aggregate decode rate across all 6 streams, about 1.1 tok/s per stream, and the 8511 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47318 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.7, 5.3, 6.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 38.3 tok/s is the aggregate decode rate across all 8 streams, about 4.8 tok/s per stream, and the 7057 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 38 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-IQ4_XS.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 40.8, 59.7, 38.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 47.1 tok/s is the aggregate decode rate across all 12 streams, about 3.9 tok/s per stream, and the 7970 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 38 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-IQ4_XS.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 47.1, 78.6, 58.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 5.5 tok/s is the aggregate decode rate across all 3 streams, about 1.8 tok/s per stream, and the 10690 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47318 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 3.9, 5.5, 6.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 68.8 tok/s is the aggregate decode rate across all 16 streams, about 4.3 tok/s per stream, and the 16014 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 68.8, 75.7, 61.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 65.6 tok/s is the aggregate decode rate across all 12 streams, about 5.5 tok/s per stream, and the 10852 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 49.1, 65.6, 49.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 38 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-IQ4_XS.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 33.9, 34.3, 31.8 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 60.1 tok/s is the aggregate decode rate across all 8 streams, about 7.5 tok/s per stream, and the 7723 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 44.4, 60.1, 49.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 42.3 tok/s is the aggregate decode rate across all 4 streams, about 10.6 tok/s per stream, and the 3957 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 42.3, 43.8, 37.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 49 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/Qwen3-Coder-Next-GGUF @ ce09c67b53bc8739eef83fe67b2f5d293c270632, file Qwen3-Coder-Next-UD-Q4_K_XL.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 40.8, 37.9, 40.6 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 20.2 tok/s is the aggregate decode rate across all 16 streams, about 1.3 tok/s per stream, and the 33238 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 113 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/GLM-4.6V-GGUF @ 7de678a269f576ae4090efbf59b558a993386fff, file GLM-4.6V-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 20.2, 19.2, 23.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 8.6 tok/s is the aggregate decode rate across all 4 streams, about 2.1 tok/s per stream, and the 3689 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 67 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47376 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF bartowski/meta-llama_Llama-4-Scout-17B-16E-Instruct-old-GGUF @ 91d32823a1855f7ca12919dffa86701254271eb9, file meta-llama_Llama-4-Scout-17B-16E-Instruct-Q4_K_M-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.6, 11.6, 12.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 14.5 tok/s is the aggregate decode rate across all 12 streams, about 1.2 tok/s per stream, and the 27321 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 113 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/GLM-4.6V-GGUF @ 7de678a269f576ae4090efbf59b558a993386fff, file GLM-4.6V-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.5, 18.8, 14.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 16.1 tok/s is the aggregate decode rate across all 8 streams, about 2.0 tok/s per stream, and the 3976 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 63 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47190 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/gpt-oss-120b-GGUF @ 7200b74bea78b0a2964ef9f3897eefa401e890ff, file gpt-oss-120b-MXFP4.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 18.6, 16.1, 18.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 7.7 tok/s is the aggregate decode rate across all 4 streams, about 1.9 tok/s per stream, and the 4815 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 77 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47380 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-Q4_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.9, 7.7, 8.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 11.2 tok/s is the aggregate decode rate across all 8 streams, about 1.4 tok/s per stream, and the 23043 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 113 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/GLM-4.6V-GGUF @ 7de678a269f576ae4090efbf59b558a993386fff, file GLM-4.6V-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 11.3, 16.3, 11.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 7.5 tok/s is the aggregate decode rate across all 2 streams, about 3.8 tok/s per stream, and the 1864 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 60 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47312 MiB of VRAM in use after load; a model download was running on the VM during this run, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-122B-A10B-GGUF @ 51eab4d59d53f573fb9206cb3ce613f1d0aa392b, file Qwen3.5-122B-A10B-UD-IQ4_XS-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.9, 7.5, 6.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 11.7 tok/s is the aggregate decode rate across all 4 streams, about 2.9 tok/s per stream, and the 12606 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 113 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF ggml-org/GLM-4.6V-GGUF @ 7de678a269f576ae4090efbf59b558a993386fff, file GLM-4.6V-Q8_0-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 13.6, 13.1, 11.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
