MTP vs DFlash: vLLM and SGLang on Qwen3.8-27B GPTQ-4bit with dual RTX 3090s

Executive summary

This report compares four speculative-decoding configurations on the same Qwen3.8-27B GPTQ-4bit target and dual-RTX-3090 host: vLLM with DFlash, SGLang with DFlash, vLLM with native MTP, and SGLang with native MTP.

For the matched single-request workload, DFlash delivered the highest decode throughput on both engines. vLLM DFlash reached 109.36 tok/s; SGLang DFlash reached 97.54 tok/s. Native MTP reached 92.78 tok/s on vLLM and 86.98 tok/s on SGLang.

• On vLLM, DFlash was 17.87% faster than MTP.
• On SGLang, DFlash was 12.14% faster than MTP.
• Within DFlash, vLLM was 12.12% faster than SGLang.
• Within MTP, vLLM was 6.67% faster than SGLang.
• SGLang delivered the lowest TTFT: 67.40 ms with DFlash and 69.30 ms with MTP.

Recommendation: use vLLM plus DFlash for maximum single-stream decode and higher offered concurrency. Use SGLang plus DFlash when low-concurrency TTFT is the priority and the workload can remain within SGLang's active-request cache limit. MTP is operational on both engines, but it did not beat DFlash in this workload.

Test setup

DFlash used four speculative draft tokens, selected by a prior vLLM sweep. MTP used two speculative tokens, matching the checkpoint's validated native-MTP configuration. The external request and measurement path are matched; the speculative methods necessarily perform different internal work.

Four-way single-request results

All values are arithmetic means over five measured iterations.

Decode ranking

1. vLLM + DFlash: 109.36 tok/s
2. SGLang + DFlash: 97.54 tok/s
3. vLLM + MTP: 92.78 tok/s
4. SGLang + MTP: 86.98 tok/s

DFlash wins within each engine. The engine gap is smaller with MTP: vLLM leads SGLang by 6.67% with MTP, versus 12.12% with DFlash.

Method comparison within each engine

MTP modestly improved vLLM TTFT, but the 4.24 ms reduction did not offset its 16.58 tok/s decode loss. On SGLang, DFlash won both decode throughput and TTFT.

Engine comparison within each method

The same engine tradeoff appears under both methods: vLLM maximizes decode throughput, while SGLang returns the first streamed token sooner.

Measurement stability

All four single-request results were stable. Decode standard deviation stayed at or below 0.15 tok/s, so the ranking is much larger than run-to-run variation.

Speculative acceptance evidence

These engine logs use different aggregation windows and rate definitions. They prove that each speculative path was active, but they are not a precise cross-engine acceptance benchmark.

MTP's higher reported acceptance percentage does not imply higher end-to-end speed. DFlash still wins because acceptance is only one term in the cost equation: draft-model execution, target verification, graph capture, memory movement, and tensor-parallel communication also contribute.

SGLang MTP correctness issue found during testing

The first SGLang MTP attempt was rejected from the comparison. Although the server command explicitly selected an unquantized MTP draft, Qwen3.5's MTP quantization resolver inherited GPTQ from the target. Seven MTP projection tensors were skipped during load, acceptance stayed at zero, and decode fell to approximately 42.3 tok/s.

The resolver was fixed to honor:

--speculative-draft-model-quantization unquant

After the fix:

• Embedded BF16 MTP tensors loaded without missing-parameter warnings.
• Decode and target-verification CUDA graphs were active.
• Acceptance recovered to approximately 1.85–2.50 tokens in sampled windows.
• SGLang MTP reached the valid 86.98 tok/s result reported above.
• The focused SGLang unit suite completed with 27 passing tests.

The invalid 42.3 tok/s attempt is not included in any comparison or ranking.

DFlash concurrency context

Only concurrency one was measured for MTP in this comparison. The existing DFlash sweep additionally measured concurrency 2, 4, 8, and 16:

SGLang DFlash peaked at concurrency four. Its Mamba state cache admitted five live requests, so offered concurrency 8 and 16 queued in waves. vLLM DFlash continued scaling through concurrency 16. No equivalent MTP concurrency sweep was run, so this section must not be read as a high-concurrency MTP comparison.

Reproduction commands

The common Localmaxxing workload was:

lmx speed-test run <vllm|sglang> \
  --mode remote \
  --base-url <endpoint> \
  --hf-id btbtyler09/Qwen3.8-27B-GPTQ-4bit \
  --served-model <served-alias> \
  --quantization GPTQ-4bit \
  --hardware hardware.json \
  --spec-method <dflash|mtp> \
  --spec-num-tokens <4|2> \
  --prompt-tokens 1044 \
  --max-tokens 512 \
  --concurrency 1 \
  --warmup 2 \
  --iterations 5 \
  --out <payload.json>

DFlash additionally supplied:

--spec-draft-model z-lab/Qwen3.8-27B-DFlash2

The corrected SGLang MTP server configuration used:

python -m sglang.launch_server \
  --model-path /models/Qwen3.8-27B-GPTQ-4bit \
  --tp-size 2 \
  --context-length 8192 \
  --mem-fraction-static 0.87 \
  --max-running-requests 4 \
  --disable-custom-all-reduce \
  --attention-backend fa3 \
  --cuda-graph-backend-prefill disabled \
  --speculative-algorithm NEXTN \
  --speculative-num-steps 2 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 3 \
  --speculative-draft-model-quantization unquant \
  --mamba-scheduler-strategy extra_buffer

The vLLM MTP server used:

vllm serve /models/Qwen3.8-27B-GPTQ-4bit \
  --tensor-parallel-size 2 \
  --max-model-len 8192 \
  --max-num-seqs 4 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching \
  --speculative-config \
  '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'

vLLM normalized the deprecated qwen3_next_mtp spelling to mtp at startup.

Artifacts and validation

The new MTP payloads are:

• qwen38-q4-mtp-vllm-k2-c1.json
• qwen38-q4-mtp-sglang-k2-c1.json

Both passed the Localmaxxing API dry-run validator with specDecoding: true, canonical specMethod: "MTP", and specNumTokens: 2. The DFlash values come from the previously validated and published matched-concurrency sweep. No benchmark payload was submitted as part of this comparison.

Conclusion

DFlash is the performance winner for this Q4 target on dual RTX 3090s. It leads native MTP by 17.87% on vLLM and 12.14% on SGLang under the matched single-request workload.

vLLM is the decode-throughput winner under either speculative method. It reaches 109.36 tok/s with DFlash and 92.78 tok/s with MTP. Its DFlash implementation also scales to 430.67 tok/s at concurrency 16 on this host.

SGLang is the latency winner at low concurrency. Its TTFT is 67.40 ms with DFlash and 69.30 ms with MTP, lower than the corresponding vLLM results. SGLang DFlash is particularly strong at concurrency four, but its five-request Mamba-cache cap causes severe queueing beyond that point.

For this model and hardware, choose vLLM + DFlash for peak decode and load scaling, or SGLang + DFlash for tightly bounded low-concurrency latency. Use MTP when avoiding a separate draft checkpoint is operationally valuable; it works correctly after the SGLang quantization fix, but it is not the fastest option measured here.