Verified 3w ago Mainly llama.cpp 200 approved runs
Submissions
2,318
200 approved public runs
Models
16
unique models
Avg output
24.0tok/s
Best 103 tok/s
Avg prefill
72.7tok/s
Best 281 tok/s
Avg total
43.4tok/s
Best 215 tok/s
Avg TTFT
10,220ms
Best 1,110 ms
Hardware
2
distinct setups
Matrix runs
36
speed test matrices
Matrix runs36+
Matrix runs 36
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 4.6 tok/s is the aggregate decode rate across all 8 streams, about 0.6 tok/s per stream, and the 6651 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.8, 4.7, 4.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 7.3 tok/s is the aggregate decode rate across all 8 streams, about 0.9 tok/s per stream, and the 33689 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.3, 6.3, 8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 6.8 tok/s is the aggregate decode rate across all 6 streams, about 1.1 tok/s per stream, and the 15955 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.3, 6.8, 6.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 2.6 tok/s is the aggregate decode rate across all 4 streams, about 0.7 tok/s per stream, and the 8894 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.9, 4.5, 2.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 8.1 tok/s is the aggregate decode rate across all 3 streams, about 2.7 tok/s per stream, and the 7962 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.2, 8.1, 8.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 6.6 tok/s is the aggregate decode rate across all 2 streams, about 3.3 tok/s per stream, and the 7416 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7, 6.6, 7.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7, 6.9, 7.1 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 5.5 tok/s is the aggregate decode rate across all 3 streams, about 1.8 tok/s per stream, and the 4291 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.9, 5.5, 2.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 9.1 tok/s is the aggregate decode rate across all 4 streams, about 2.3 tok/s per stream, and the 9610 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 4 server slots of 4096 tokens each, context 16384. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.1, 7.9, 6.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 8.8 tok/s is the aggregate decode rate across all 2 streams, about 4.4 tok/s per stream, and the 3691 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.9, 8.8, 11.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 6.5 tok/s is the aggregate decode rate across all 3 streams, about 2.2 tok/s per stream, and the 9998 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 4 server slots of 4096 tokens each, context 16384. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.4, 8.3, 6.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 9.2 tok/s is the aggregate decode rate across all 2 streams, about 4.6 tok/s per stream, and the 5597 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 4 server slots of 4096 tokens each, context 16384. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.2, 9.3, 9.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 10.2, 10.5, 10.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 4 server slots of 4096 tokens each, context 16384. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7.1, 7, 6.9 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 12.3 tok/s is the aggregate decode rate across all 16 streams, about 0.8 tok/s per stream, and the 46128 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.7, 12.2, 12.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 7 tok/s is the aggregate decode rate across all 6 streams, about 1.2 tok/s per stream, and the 8324 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 129 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 46918 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/MiniMax-M3-GGUF @ abdce34b5ab2e7dd9260130415c01a1f25561210, file MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7, 7, 4.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 8.6 tok/s is the aggregate decode rate across all 12 streams, about 0.7 tok/s per stream, and the 54654 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.5, 9.2, 8.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 5 tok/s is the aggregate decode rate across all 3 streams, about 1.7 tok/s per stream, and the 5595 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 129 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 46918 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/MiniMax-M3-GGUF @ abdce34b5ab2e7dd9260130415c01a1f25561210, file MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.9, 6.4, 5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 5 tok/s is the aggregate decode rate across all 2 streams, about 2.5 tok/s per stream, and the 4120 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 129 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 46918 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/MiniMax-M3-GGUF @ abdce34b5ab2e7dd9260130415c01a1f25561210, file MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5, 5.2, 2.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 129 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 46918 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/MiniMax-M3-GGUF @ abdce34b5ab2e7dd9260130415c01a1f25561210, file MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 6.6, 6.6, 6.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 3.9 tok/s is the aggregate decode rate across all 4 streams, about 1.0 tok/s per stream, and the 8125 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47406 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768, KV cache q8_0 keys / q8_0 values (-ctk/-ctv). GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.5, 4.8, 3.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 6.6 tok/s is the aggregate decode rate across all 6 streams, about 1.1 tok/s per stream, and the 15719 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.6, 7.2, 6.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47406 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768, KV cache q8_0 keys / q8_0 values (-ctk/-ctv). GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 21.2, 20, 20.2 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 7.3 tok/s is the aggregate decode rate across all 4 streams, about 1.8 tok/s per stream, and the 8434 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47800 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768, KV cache q8_0 keys / q8_0 values (-ctk/-ctv). GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.7, 8.6, 7.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 6 tok/s is the aggregate decode rate across all 4 streams, about 1.5 tok/s per stream, and the 13113 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.3, 6, 8.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 4.6 tok/s is the aggregate decode rate across all 8 streams, about 0.6 tok/s per stream, and the 6651 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.8, 4.7, 4.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 7.3 tok/s is the aggregate decode rate across all 8 streams, about 0.9 tok/s per stream, and the 33689 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.3, 6.3, 8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 6.8 tok/s is the aggregate decode rate across all 6 streams, about 1.1 tok/s per stream, and the 15955 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.3, 6.8, 6.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 2.6 tok/s is the aggregate decode rate across all 4 streams, about 0.7 tok/s per stream, and the 8894 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.9, 4.5, 2.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 8.1 tok/s is the aggregate decode rate across all 3 streams, about 2.7 tok/s per stream, and the 7962 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.2, 8.1, 8.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 6.6 tok/s is the aggregate decode rate across all 2 streams, about 3.3 tok/s per stream, and the 7416 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7, 6.6, 7.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7, 6.9, 7.1 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 5.5 tok/s is the aggregate decode rate across all 3 streams, about 1.8 tok/s per stream, and the 4291 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.9, 5.5, 2.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 9.1 tok/s is the aggregate decode rate across all 4 streams, about 2.3 tok/s per stream, and the 9610 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 4 server slots of 4096 tokens each, context 16384. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.1, 7.9, 6.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 8.8 tok/s is the aggregate decode rate across all 2 streams, about 4.4 tok/s per stream, and the 3691 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.9, 8.8, 11.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 6.5 tok/s is the aggregate decode rate across all 3 streams, about 2.2 tok/s per stream, and the 9998 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 4 server slots of 4096 tokens each, context 16384. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.4, 8.3, 6.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 9.2 tok/s is the aggregate decode rate across all 2 streams, about 4.6 tok/s per stream, and the 5597 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 4 server slots of 4096 tokens each, context 16384. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.2, 9.3, 9.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 114 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47546 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3.5-397B-A17B-GGUF @ da33c16fa4440f831149fcf53b98a22bc07785e5, file Qwen3.5-397B-A17B-UD-IQ2_XXS-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 10.2, 10.5, 10.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 4 server slots of 4096 tokens each, context 16384. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7.1, 7, 6.9 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 16 requests sent at once (lmx --concurrency 16); 12.3 tok/s is the aggregate decode rate across all 16 streams, about 0.8 tok/s per stream, and the 46128 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.7, 12.2, 12.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 7 tok/s is the aggregate decode rate across all 6 streams, about 1.2 tok/s per stream, and the 8324 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 129 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 46918 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/MiniMax-M3-GGUF @ abdce34b5ab2e7dd9260130415c01a1f25561210, file MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7, 7, 4.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 12 requests sent at once (lmx --concurrency 12); 8.6 tok/s is the aggregate decode rate across all 12 streams, about 0.7 tok/s per stream, and the 54654 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.5, 9.2, 8.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 5 tok/s is the aggregate decode rate across all 3 streams, about 1.7 tok/s per stream, and the 5595 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 129 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 46918 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/MiniMax-M3-GGUF @ abdce34b5ab2e7dd9260130415c01a1f25561210, file MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.9, 6.4, 5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 5 tok/s is the aggregate decode rate across all 2 streams, about 2.5 tok/s per stream, and the 4120 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 129 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 46918 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/MiniMax-M3-GGUF @ abdce34b5ab2e7dd9260130415c01a1f25561210, file MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5, 5.2, 2.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 129 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 46918 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/MiniMax-M3-GGUF @ abdce34b5ab2e7dd9260130415c01a1f25561210, file MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 6.6, 6.6, 6.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 3.9 tok/s is the aggregate decode rate across all 4 streams, about 1.0 tok/s per stream, and the 8125 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47406 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768, KV cache q8_0 keys / q8_0 values (-ctk/-ctv). GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.5, 4.8, 3.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 6.6 tok/s is the aggregate decode rate across all 6 streams, about 1.1 tok/s per stream, and the 15719 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.6, 7.2, 6.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47406 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768, KV cache q8_0 keys / q8_0 values (-ctk/-ctv). GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 21.2, 20, 20.2 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 7.3 tok/s is the aggregate decode rate across all 4 streams, about 1.8 tok/s per stream, and the 8434 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47800 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768, KV cache q8_0 keys / q8_0 values (-ctk/-ctv). GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.7, 8.6, 7.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 6 tok/s is the aggregate decode rate across all 4 streams, about 1.5 tok/s per stream, and the 13113 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 96 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 16 server slots of 4096 tokens each, context 65536. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-Q2_K_XL-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.3, 6, 8.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
