Reports

Community-authored local inference reports

Published field reportsnewest

Strata dual-16 GB (5080+5060 Ti): verified 98.1/102.8 tok/s, 262k context, 3 full evals — Unsloth UD-Q4_K_XL

Complete field report on Strata v0.1.42–v0.1.43: 15 verified speed/depth rows (canonical prompts, cold prefill, MTP), and three full quality evals — HellaSwag 92.66%, GSM8K 98.10%, ARC-Challenge 98.43% — all 13/13 shards approved. 180B MoE on 2x16 GB consumer GPUs via expert streaming.

Q4_K_XL custom2× Multi-GPU · 32 GB

15 speed tests

Strata dual-16 GB (5080+5060 Ti): verified 105.7/107.6 tok/s, 262k context, cold prefill 3.2–3.7k — UD-IQ4_XS

Qwen3.8-Flash-Next-GGUF

Verified UD-IQ4_XS speed suite on Strata v0.1.43.1: reasoning 105.7 / code 107.6 tok/s (TTFT 649/589 ms), verified-at-depth decode 101.4/100.8/94.3 tok/s at 16k/65k/131k with cold prefill 3.2–3.7k tok/s. All five runs verifiedRun:true; full contamination-handling disclosure included.

IQ4_XS custom2× Multi-GPU · 32 GB

5 speed tests

rtx-5080 multi-gpu speculative-decoding expert-streaming moe long-context

DeepSeek-V4.1-Flash on 4x DGX Spark: 76 tok/s verified

DeepSeek-V4.1-Flash

Verified single-stream run of deepseek-ai/DeepSeek-V4.1-Flash on 4x NVIDIA GB10 (DGX Spark, 512 GB unified, TP4, SGLang, DSPARK spec decoding): 76.0 tok/s decode, 887.6 tok/s prefill, 308.7 ms TTFT, 423 GB unified memory in use, 37.2% draft acceptance.

FP8+MXFP4 sglangGB10 Grace Blackwell · 512 GB

1 speed test

throughput multi-gpu unified-memory sglang moe speculative-decoding dgx-spark fp8

Qwen3.6-35B-A3B on dual RTX 3060 12GB (llama.cpp tensor + MTP, 88k)

Qwen3.6-35B-A3B-GGUF

Two RTX 3060 12GB, no NVLink. llama.cpp b10682, Unsloth UD-Q4_K_XL, 88k. Tensor split + native MTP + q4_0 draft KV: 129 tok/s code / 104 tok/s prose. 96k+MTP does not fit.

1 speed test

throughput multi-gpu rtx-3060 llama.cpp qwen3.6 moe