Verified 3w ago Mainly llama.cpp 200 approved runs
Submissions
2,318
200 approved public runs
Models
16
unique models
Avg output
24.0tok/s
Best 103 tok/s
Avg prefill
72.7tok/s
Best 281 tok/s
Avg total
43.4tok/s
Best 215 tok/s
Avg TTFT
10,220ms
Best 1,110 ms
Hardware
2
distinct setups
Matrix runs
36
speed test matrices
Matrix runs36+
Matrix runs 36
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 13.9 tok/s is the aggregate decode rate across all 6 streams, about 2.3 tok/s per stream, and the 22076 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 14.7, 13.9, 14 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 5 requests sent at once (lmx --concurrency 5); 14.3 tok/s is the aggregate decode rate across all 5 streams, about 2.9 tok/s per stream, and the 21151 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 14.3, 13.6, 14.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 13.2 tok/s is the aggregate decode rate across all 4 streams, about 3.3 tok/s per stream, and the 18910 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.7, 13.6, 13.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 8.9 tok/s is the aggregate decode rate across all 3 streams, about 3.0 tok/s per stream, and the 18667 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.8, 10.4, 8.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 8.1 tok/s is the aggregate decode rate across all 2 streams, about 4.0 tok/s per stream, and the 10488 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.1, 9, 9.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 10.8, 10.7, 11.7 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 10 tok/s is the aggregate decode rate across all 8 streams, about 1.2 tok/s per stream, and the 34358 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.3, 10, 10.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 7 requests sent at once (lmx --concurrency 7); 9.7 tok/s is the aggregate decode rate across all 7 streams, about 1.4 tok/s per stream, and the 33638 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.7, 7.8, 9.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 7.2 tok/s is the aggregate decode rate across all 6 streams, about 1.2 tok/s per stream, and the 32936 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.4, 7.4, 7.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 5 requests sent at once (lmx --concurrency 5); 6.8 tok/s is the aggregate decode rate across all 5 streams, about 1.4 tok/s per stream, and the 17603 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.9, 6.8, 6.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 6.1 tok/s is the aggregate decode rate across all 4 streams, about 1.5 tok/s per stream, and the 15230 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.6, 5.3, 6.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 4.2 tok/s is the aggregate decode rate across all 3 streams, about 1.4 tok/s per stream, and the 14178 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4, 5.6, 4.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 4.4 tok/s is the aggregate decode rate across all 2 streams, about 2.2 tok/s per stream, and the 8038 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.6, 5.1, 4.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 4.9, 5, 5.1 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.62 VRAM_RESERVE_MIB=1280 MAX_CONTEXT=65536 CONVERSATION_CACHE_MIB=4096. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54.2, 53.4, 57.7 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.62 VRAM_RESERVE_MIB=1280 MAX_CONTEXT=131072 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 52.2, 57.5, 59.4 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1536 MAX_CONTEXT=65536 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54.1, 52.9, 52.5 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1536 MAX_CONTEXT=131072 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 51.7, 56.9, 57.3 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.58 VRAM_RESERVE_MIB=1280 MAX_CONTEXT=131072 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 51.4, 56.6, 58.6 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.58 VRAM_RESERVE_MIB=1536 MAX_CONTEXT=131072 CONVERSATION_CACHE_MIB=4096. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54.4, 58, 59.1 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1280 MAX_CONTEXT=65536 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54, 54.8, 58.9 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.62 VRAM_RESERVE_MIB=1536 MAX_CONTEXT=65536 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54.6, 56.8, 56.2 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 24.8 tok/s is the aggregate decode rate across all 8 streams, about 3.1 tok/s per stream, and the 5284 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 88 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47488 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF @ 437d6915c5c512869e4cf8b16b840bbe3fb172bc, file Qwen3-235B-A22B-Instruct-2507-UD-Q2_K_XL-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 25, 24.8, 24.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 20.7 tok/s is the aggregate decode rate across all 6 streams, about 3.4 tok/s per stream, and the 6501 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 88 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47488 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF @ 437d6915c5c512869e4cf8b16b840bbe3fb172bc, file Qwen3-235B-A22B-Instruct-2507-UD-Q2_K_XL-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 21.6, 20.7, 19.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 19.6 tok/s is the aggregate decode rate across all 4 streams, about 4.9 tok/s per stream, and the 4560 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 88 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47488 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF @ 437d6915c5c512869e4cf8b16b840bbe3fb172bc, file Qwen3-235B-A22B-Instruct-2507-UD-Q2_K_XL-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 14.7, 19.6, 15.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 13.9 tok/s is the aggregate decode rate across all 6 streams, about 2.3 tok/s per stream, and the 22076 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 14.7, 13.9, 14 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 5 requests sent at once (lmx --concurrency 5); 14.3 tok/s is the aggregate decode rate across all 5 streams, about 2.9 tok/s per stream, and the 21151 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 14.3, 13.6, 14.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 13.2 tok/s is the aggregate decode rate across all 4 streams, about 3.3 tok/s per stream, and the 18910 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.7, 13.6, 13.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 8.9 tok/s is the aggregate decode rate across all 3 streams, about 3.0 tok/s per stream, and the 18667 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.8, 10.4, 8.9 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 8.1 tok/s is the aggregate decode rate across all 2 streams, about 4.0 tok/s per stream, and the 10488 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.1, 9, 9.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 10.8, 10.7, 11.7 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 10 tok/s is the aggregate decode rate across all 8 streams, about 1.2 tok/s per stream, and the 34358 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.3, 10, 10.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 7 requests sent at once (lmx --concurrency 7); 9.7 tok/s is the aggregate decode rate across all 7 streams, about 1.4 tok/s per stream, and the 33638 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 9.7, 7.8, 9.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 7.2 tok/s is the aggregate decode rate across all 6 streams, about 1.2 tok/s per stream, and the 32936 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 7.4, 7.4, 7.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 5 requests sent at once (lmx --concurrency 5); 6.8 tok/s is the aggregate decode rate across all 5 streams, about 1.4 tok/s per stream, and the 17603 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.9, 6.8, 6.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 6.1 tok/s is the aggregate decode rate across all 4 streams, about 1.5 tok/s per stream, and the 15230 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.6, 5.3, 6.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 4.2 tok/s is the aggregate decode rate across all 3 streams, about 1.4 tok/s per stream, and the 14178 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4, 5.6, 4.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 4.4 tok/s is the aggregate decode rate across all 2 streams, about 2.2 tok/s per stream, and the 8038 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 4.6, 5.1, 4.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 108 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-Q2_K_XL-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 4.9, 5, 5.1 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.62 VRAM_RESERVE_MIB=1280 MAX_CONTEXT=65536 CONVERSATION_CACHE_MIB=4096. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54.2, 53.4, 57.7 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.62 VRAM_RESERVE_MIB=1280 MAX_CONTEXT=131072 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 52.2, 57.5, 59.4 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1536 MAX_CONTEXT=65536 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54.1, 52.9, 52.5 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1536 MAX_CONTEXT=131072 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 51.7, 56.9, 57.3 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.58 VRAM_RESERVE_MIB=1280 MAX_CONTEXT=131072 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 51.4, 56.6, 58.6 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.58 VRAM_RESERVE_MIB=1536 MAX_CONTEXT=131072 CONVERSATION_CACHE_MIB=4096. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54.4, 58, 59.1 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1280 MAX_CONTEXT=65536 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54, 54.8, 58.9 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:6f03dde70323). Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f920; the GPU holds the dense layers and a cache of hot experts, the CPU computes the rest of the experts from system RAM at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.62 VRAM_RESERVE_MIB=1536 MAX_CONTEXT=65536 CONVERSATION_CACHE_MIB=2048. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 54.6, 56.8, 56.2 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-27-b65-vm-localmaxxing. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 24.8 tok/s is the aggregate decode rate across all 8 streams, about 3.1 tok/s per stream, and the 5284 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 88 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47488 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF @ 437d6915c5c512869e4cf8b16b840bbe3fb172bc, file Qwen3-235B-A22B-Instruct-2507-UD-Q2_K_XL-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 25, 24.8, 24.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 20.7 tok/s is the aggregate decode rate across all 6 streams, about 3.4 tok/s per stream, and the 6501 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 88 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47488 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF @ 437d6915c5c512869e4cf8b16b840bbe3fb172bc, file Qwen3-235B-A22B-Instruct-2507-UD-Q2_K_XL-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 21.6, 20.7, 19.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 19.6 tok/s is the aggregate decode rate across all 4 streams, about 4.9 tok/s per stream, and the 4560 ms time to first token is under that load. KASA cuda-vm: 1 x NVIDIA RTX A6000 48 GB (PCIe passthrough, driver 615.71.10) in a VM with 16 vCPU pinned to 8 of the 16 cores (16 threads) of an AMD EPYC 7343 (another VM holds the other 8 cores) and 128 GiB. llama.cpp llama-server b11889 (CUDA image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 88 GB model does not fit the 48 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU (loaded with -lm none, so the CPU part is in ordinary RAM, not memory-mapped); the exact split is not logged at the default verbosity; 47488 MiB of VRAM in use after load, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF @ 437d6915c5c512869e4cf8b16b840bbe3fb172bc, file Qwen3-235B-A22B-Instruct-2507-UD-Q2_K_XL-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 14.7, 19.6, 15.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
