Reports

Community-authored local inference reports

Published field reportsnewest

Zeus 2× Arc Pro B70: llama.cpp SYCL FP16 tuning across Qwen3.8 and nine other model routes

On two Intel Arc Pro B70s, a b11190 SYCL FP16 build substantially improved prompt processing across 10 production routes. This report gives matched serving results, copy-heavy generation gains from n-gram drafting, rejected tuning variants, and measurement limits.

Q6_K_XL focus; Q8_0, Q4_K_M, UD-Q4_K_XL llama.cpp b11190, SYCL / Level Zero

DeepSeek-V4.1-Flash on 4x DGX Spark: 76 tok/s verified

DeepSeek-V4.1-Flash

Verified single-stream run of deepseek-ai/DeepSeek-V4.1-Flash on 4x NVIDIA GB10 (DGX Spark, 512 GB unified, TP4, SGLang, DSPARK spec decoding): 76.0 tok/s decode, 887.6 tok/s prefill, 308.7 ms TTFT, 423 GB unified memory in use, 37.2% draft acceptance.

FP8+MXFP4 sglangGB10 Grace Blackwell · 512 GB

1 benchmark

throughput multi-gpu unified-memory sglang moe speculative-decoding dgx-spark fp8

Qwen3.6-35B-A3B on dual RTX 3060 12GB (llama.cpp tensor + MTP, 88k)

Qwen3.6-35B-A3B-GGUF

Two RTX 3060 12GB, no NVLink. llama.cpp b10682, Unsloth UD-Q4_K_XL, 88k. Tensor split + native MTP + q4_0 draft KV: 129 tok/s code / 104 tok/s prose. 96k+MTP does not fit.

1 benchmark

throughput multi-gpu rtx-3060 llama.cpp qwen3.6 moe

Qwen3.8-27B on dual RTX 3060 12GB (llama.cpp tensor + MTP)

Qwen3.8-27B-GGUF

Two RTX 3060 12GB, no NVLink. llama.cpp b10682, Unsloth UD-Q4_K_XL, 64k text-only. Tensor split + native MTP: 57 tok/s code / 46 tok/s prose. DFLASH2 on layer split lost.

1 benchmark

throughput multi-gpu rtx-3060 llama.cpp qwen3.8