Verified 3w ago Mainly llama.cpp 200 approved runs
Submissions
2,318
200 approved public runs
Models
16
unique models
Avg output
24.0tok/s
Best 103 tok/s
Avg prefill
72.7tok/s
Best 281 tok/s
Avg total
43.4tok/s
Best 215 tok/s
Avg TTFT
10,220ms
Best 1,110 ms
Hardware
2
distinct setups
Matrix runs
36
speed test matrices
Matrix runs36+
Matrix runs 36
A matrix run measures prompt reading (prefill) and generation (decode) speed in one go, across several context lengths and numbers of simultaneous requests. Showing the newest 10.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 44.6 tok/s is the aggregate decode rate across all 4 streams, about 11.2 tok/s per stream. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:632690be0d76), its entrypoint adapted for XeStrata's Unsloth model family (pack with --compat-bf16, no vision projector). Weights: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS @ 38bb39ee (byte-identical to 766911a6); the GPU holds the dense layers and a cache of hot experts, 55 GiB of experts (all of them) are kept in system RAM and the CPU computes the rest of the experts at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1024 MAX_CONTEXT=65536 PCIE_FRAC=0.5. Canonical reasoning-v1 prompt with a unique nonce; one discarded warm-up batch, then 3 measured batches: 44.9, 45.2, 44.6 tok/s; this is the median. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine. Image rebuilt 2026-10-30 from the same recipe commit, XeStrata package (SHA-256 pinned) and base digest; its id differs from the 2026-10-27 IQ3_S runs' image (sha256:6f03dde7) because the Intel GPU runtime and oneMKL come from unpinned package repositories.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 48.7 tok/s is the aggregate decode rate across all 2 streams, about 24.4 tok/s per stream. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:632690be0d76), its entrypoint adapted for XeStrata's Unsloth model family (pack with --compat-bf16, no vision projector). Weights: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS @ 38bb39ee (byte-identical to 766911a6); the GPU holds the dense layers and a cache of hot experts, 55 GiB of experts (all of them) are kept in system RAM and the CPU computes the rest of the experts at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1024 MAX_CONTEXT=65536 PCIE_FRAC=0.5. Canonical reasoning-v1 prompt with a unique nonce; one discarded warm-up batch, then 3 measured batches: 48.7, 48.6, 50.3 tok/s; this is the median. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine. Image rebuilt 2026-10-30 from the same recipe commit, XeStrata package (SHA-256 pinned) and base digest; its id differs from the 2026-10-27 IQ3_S runs' image (sha256:6f03dde7) because the Intel GPU runtime and oneMKL come from unpinned package repositories.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:632690be0d76), its entrypoint adapted for XeStrata's Unsloth model family (pack with --compat-bf16, no vision projector). Weights: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS @ 38bb39ee (byte-identical to 766911a6); the GPU holds the dense layers and a cache of hot experts, 55 GiB of experts (all of them) are kept in system RAM and the CPU computes the rest of the experts at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: recipe defaults. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 53.8, 57.5, 58.1 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-30-b65-vm-xestrata-4bit. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine. Image rebuilt 2026-10-30 from the same recipe commit, XeStrata package (SHA-256 pinned) and base digest; its id differs from the 2026-10-27 IQ3_S runs' image (sha256:6f03dde7) because the Intel GPU runtime and oneMKL come from unpinned package repositories.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:632690be0d76), its entrypoint adapted for XeStrata's Unsloth model family (pack with --compat-bf16, no vision projector). Weights: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS @ 38bb39ee (byte-identical to 766911a6); the GPU holds the dense layers and a cache of hot experts, 55 GiB of experts (all of them) are kept in system RAM and the CPU computes the rest of the experts at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1024 MAX_CONTEXT=65536 PCIE_FRAC=0.5. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 52.9, 62.4, 62.7 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-30-b65-vm-xestrata-4bit. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine. Image rebuilt 2026-10-30 from the same recipe commit, XeStrata package (SHA-256 pinned) and base digest; its id differs from the 2026-10-27 IQ3_S runs' image (sha256:6f03dde7) because the Intel GPU runtime and oneMKL come from unpinned package repositories.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 5.2 tok/s is the aggregate decode rate across all 3 streams, about 1.7 tok/s per stream, and the 13821 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.2, 7.4, 5.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 6.4 tok/s is the aggregate decode rate across all 2 streams, about 3.2 tok/s per stream, and the 8489 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.5, 6.4, 6.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 5.2, 4.7, 5.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 13.8 tok/s is the aggregate decode rate across all 8 streams, about 1.7 tok/s per stream, and the 29619 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 13.6, 13.8, 13.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 7 requests sent at once (lmx --concurrency 7); 12.8 tok/s is the aggregate decode rate across all 7 streams, about 1.8 tok/s per stream, and the 27038 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.8, 10.9, 13.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 10.7 tok/s is the aggregate decode rate across all 6 streams, about 1.8 tok/s per stream, and the 14282 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.9, 10.6, 10.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 5 requests sent at once (lmx --concurrency 5); 10 tok/s is the aggregate decode rate across all 5 streams, about 2.0 tok/s per stream, and the 12956 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.2, 9.9, 10 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 8.7 tok/s is the aggregate decode rate across all 4 streams, about 2.2 tok/s per stream, and the 11345 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.7, 9, 8.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 6.6 tok/s is the aggregate decode rate across all 3 streams, about 2.2 tok/s per stream, and the 9659 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.2, 7.2, 6.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 6.2 tok/s is the aggregate decode rate across all 2 streams, about 3.1 tok/s per stream, and the 6157 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.2, 6.4, 6.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 31.6 tok/s is the aggregate decode rate across all 8 streams, about 4.0 tok/s per stream, and the 19746 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 31.1, 31.6, 31.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7.4, 7.3, 7.4 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 7 requests sent at once (lmx --concurrency 7); 29 tok/s is the aggregate decode rate across all 7 streams, about 4.1 tok/s per stream, and the 11004 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 29, 25.9, 29.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 24.6 tok/s is the aggregate decode rate across all 6 streams, about 4.1 tok/s per stream, and the 17789 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 25.2, 24.6, 23.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 5 requests sent at once (lmx --concurrency 5); 22.5 tok/s is the aggregate decode rate across all 5 streams, about 4.5 tok/s per stream, and the 16834 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 22.1, 22.8, 22.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 12.5 tok/s is the aggregate decode rate across all 4 streams, about 3.1 tok/s per stream, and the 16965 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.5, 12.7, 14 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 13.2 tok/s is the aggregate decode rate across all 3 streams, about 4.4 tok/s per stream, and the 15866 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 13.1, 14.8, 13.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 12.2 tok/s is the aggregate decode rate across all 2 streams, about 6.1 tok/s per stream, and the 8320 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 11.7, 13.2, 12.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 16.2, 17.4, 15.9 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 16 tok/s is the aggregate decode rate across all 8 streams, about 2.0 tok/s per stream, and the 23443 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 16, 16.2, 16.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 7 requests sent at once (lmx --concurrency 7); 16.8 tok/s is the aggregate decode rate across all 7 streams, about 2.4 tok/s per stream, and the 17449 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 16.8, 15, 16.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 44.6 tok/s is the aggregate decode rate across all 4 streams, about 11.2 tok/s per stream. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:632690be0d76), its entrypoint adapted for XeStrata's Unsloth model family (pack with --compat-bf16, no vision projector). Weights: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS @ 38bb39ee (byte-identical to 766911a6); the GPU holds the dense layers and a cache of hot experts, 55 GiB of experts (all of them) are kept in system RAM and the CPU computes the rest of the experts at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1024 MAX_CONTEXT=65536 PCIE_FRAC=0.5. Canonical reasoning-v1 prompt with a unique nonce; one discarded warm-up batch, then 3 measured batches: 44.9, 45.2, 44.6 tok/s; this is the median. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine. Image rebuilt 2026-10-30 from the same recipe commit, XeStrata package (SHA-256 pinned) and base digest; its id differs from the 2026-10-27 IQ3_S runs' image (sha256:6f03dde7) because the Intel GPU runtime and oneMKL come from unpinned package repositories.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 48.7 tok/s is the aggregate decode rate across all 2 streams, about 24.4 tok/s per stream. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:632690be0d76), its entrypoint adapted for XeStrata's Unsloth model family (pack with --compat-bf16, no vision projector). Weights: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS @ 38bb39ee (byte-identical to 766911a6); the GPU holds the dense layers and a cache of hot experts, 55 GiB of experts (all of them) are kept in system RAM and the CPU computes the rest of the experts at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1024 MAX_CONTEXT=65536 PCIE_FRAC=0.5. Canonical reasoning-v1 prompt with a unique nonce; one discarded warm-up batch, then 3 measured batches: 48.7, 48.6, 50.3 tok/s; this is the median. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine. Image rebuilt 2026-10-30 from the same recipe commit, XeStrata package (SHA-256 pinned) and base digest; its id differs from the 2026-10-27 IQ3_S runs' image (sha256:6f03dde7) because the Intel GPU runtime and oneMKL come from unpinned package repositories.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:632690be0d76), its entrypoint adapted for XeStrata's Unsloth model family (pack with --compat-bf16, no vision projector). Weights: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS @ 38bb39ee (byte-identical to 766911a6); the GPU holds the dense layers and a cache of hot experts, 55 GiB of experts (all of them) are kept in system RAM and the CPU computes the rest of the experts at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: recipe defaults. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 53.8, 57.5, 58.1 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-30-b65-vm-xestrata-4bit. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine. Image rebuilt 2026-10-30 from the same recipe commit, XeStrata package (SHA-256 pinned) and base digest; its id differs from the 2026-10-27 IQ3_S runs' image (sha256:6f03dde7) because the Intel GPU runtime and oneMKL come from unpinned package repositories.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. Engine: XeStrata xe0.1.40.2.2 (MistVVK/XeStrata, the Intel fork of Strata; LGPL-3.0) in the chriscorbell/intel-arc-pro-b70-qwen38-flash-next recipe image built from commit 6aad3506 (local image local/xestrata-qwen38:xe0.1.40.2.2, sha256:632690be0d76), its entrypoint adapted for XeStrata's Unsloth model family (pack with --compat-bf16, no vision projector). Weights: unsloth/Qwen3.8-Flash-Next-GGUF UD-IQ4_XS @ 38bb39ee (byte-identical to 766911a6); the GPU holds the dense layers and a cache of hot experts, 55 GiB of experts (all of them) are kept in system RAM and the CPU computes the rest of the experts at the same time. MTP speculative decoding (4 tokens), int8 KV cache. Settings: SPEC_MIN_P=0.6 VRAM_RESERVE_MIB=1024 MAX_CONTEXT=65536 PCIE_FRAC=0.5. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 52.9, 62.4, 62.7 tok/s; this is the median (tie to the earlier). Campaign ai/vm/b65-vm/2026-10-30-b65-vm-xestrata-4bit. Prompt/output samples were added from the lmx run file; engineTimingsRaw is XeStrata's own per-request log (decode tokens and tok/s, expert-cache hit rate), because lmx does not capture it for this engine. Image rebuilt 2026-10-30 from the same recipe commit, XeStrata package (SHA-256 pinned) and base digest; its id differs from the 2026-10-27 IQ3_S runs' image (sha256:6f03dde7) because the Intel GPU runtime and oneMKL come from unpinned package repositories.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 5.2 tok/s is the aggregate decode rate across all 3 streams, about 1.7 tok/s per stream, and the 13821 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 5.2, 7.4, 5.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 6.4 tok/s is the aggregate decode rate across all 2 streams, about 3.2 tok/s per stream, and the 8489 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.5, 6.4, 6.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 120 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/GLM-5.3-Flash-GGUF @ a38483c8cd5df544f53d70fb281afe97369d5ab6, file GLM-5.3-Flash-UD-IQ3_XXS-00001-of-00004.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 5.2, 4.7, 5.5 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 13.8 tok/s is the aggregate decode rate across all 8 streams, about 1.7 tok/s per stream, and the 29619 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 13.6, 13.8, 13.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 7 requests sent at once (lmx --concurrency 7); 12.8 tok/s is the aggregate decode rate across all 7 streams, about 1.8 tok/s per stream, and the 27038 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.8, 10.9, 13.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 10.7 tok/s is the aggregate decode rate across all 6 streams, about 1.8 tok/s per stream, and the 14282 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.9, 10.6, 10.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 5 requests sent at once (lmx --concurrency 5); 10 tok/s is the aggregate decode rate across all 5 streams, about 2.0 tok/s per stream, and the 12956 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 10.2, 9.9, 10 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 8.7 tok/s is the aggregate decode rate across all 4 streams, about 2.2 tok/s per stream, and the 11345 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 8.7, 9, 8.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 6.6 tok/s is the aggregate decode rate across all 3 streams, about 2.2 tok/s per stream, and the 9659 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.2, 7.2, 6.6 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 6.2 tok/s is the aggregate decode rate across all 2 streams, about 3.1 tok/s per stream, and the 6157 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 6.2, 6.4, 6.4 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 31.6 tok/s is the aggregate decode rate across all 8 streams, about 4.0 tok/s per stream, and the 19746 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 31.1, 31.6, 31.3 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 1 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 90 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF unsloth/DeepSeek-V4-Flash-0731-GGUF @ fbbb5b93fb787c21338159b0af3318bb3f4d9768, file DeepSeek-V4-Flash-0731-UD-IQ2_M-00001-of-00003.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 7.4, 7.3, 7.4 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 7 requests sent at once (lmx --concurrency 7); 29 tok/s is the aggregate decode rate across all 7 streams, about 4.1 tok/s per stream, and the 11004 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 29, 25.9, 29.7 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 6 requests sent at once (lmx --concurrency 6); 24.6 tok/s is the aggregate decode rate across all 6 streams, about 4.1 tok/s per stream, and the 17789 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 25.2, 24.6, 23.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 5 requests sent at once (lmx --concurrency 5); 22.5 tok/s is the aggregate decode rate across all 5 streams, about 4.5 tok/s per stream, and the 16834 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 22.1, 22.8, 22.5 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 4 requests sent at once (lmx --concurrency 4); 12.5 tok/s is the aggregate decode rate across all 4 streams, about 3.1 tok/s per stream, and the 16965 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 12.5, 12.7, 14 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 3 requests sent at once (lmx --concurrency 3); 13.2 tok/s is the aggregate decode rate across all 3 streams, about 4.4 tok/s per stream, and the 15866 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 13.1, 14.8, 13.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 2 requests sent at once (lmx --concurrency 2); 12.2 tok/s is the aggregate decode rate across all 2 streams, about 6.1 tok/s per stream, and the 8320 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 11.7, 13.2, 12.2 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 118 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/Step-3.5-Flash-GGUF @ c439ad64d667632f3e1507757cbe225b6a46e6b8, file Step-3.5-Flash-Q4_K.gguf. Canonical reasoning-v1, lmx single-shot requests after one discarded warm-up. Measured requests: 16.2, 17.4, 15.9 tok/s; this is the median (tie to the earlier). Prompt and output samples were added from the lmx run file, engineTimingsRaw from exact llama-server /metrics deltas around the request.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 8 requests sent at once (lmx --concurrency 8); 16 tok/s is the aggregate decode rate across all 8 streams, about 2.0 tok/s per stream, and the 23443 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 16, 16.2, 16.1 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1. Selection rule applied to every run in this round: the median of the 2 measured requests whose lmx promptTokens is a whole number; the others report a fractional median, which LocalMaxxing refuses.
CONCURRENCY TEST, NOT A SINGLE-USER RESULT. 7 requests sent at once (lmx --concurrency 7); 16.8 tok/s is the aggregate decode rate across all 7 streams, about 2.4 tok/s per stream, and the 17449 ms time to first token is under that load. KASA b65-vm: 1 x Intel Arc Pro B65 32 GB (PCIe passthrough, xe driver, Ubuntu 26.04 kernel 7.0) in a VM with 16 vCPU pinned to 8 cores (16 threads) of an AMD EPYC 7343 and 128 GiB. llama.cpp llama-server b11913 (image ghcr.io/ggml-org/llama.cpp), placement by llama.cpp's default --fit (1 GiB margin): the 126 GB model does not fit the 32 GB GPU, so llama.cpp's --fit kept the layers on the GPU and placed part of the MoE expert weights in system RAM on the CPU; the exact split is not logged at the default verbosity, 8 generation threads, 16 batch threads, flash attention on, 8 server slots of 4096 tokens each, context 32768. GGUF ggml-org/MiMo-V2.6-Flash-RL-GGUF @ f1ef9bfdc3706af3f807752765e52b885e113aae, file MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf. Each request: LocalMaxxing's canonical reasoning-v1 prompt with a unique nonce, 256 output tokens. One discarded warm-up batch, then 3 measured batches: 16.8, 15, 16.8 tok/s; this is the median. Not eligible for the Verified badge, which requires batch 1.
