vLLM vs SGLang: Qwen3.8-27B GPTQ-4bit with DFlash2 on dual RTX 3090s

Executive summary

This report compares vLLM and SGLang under the same external workload: the same Qwen3.8-27B GPTQ-4bit target, official Z-Lab DFlash2 draft, two RTX 3090 GPUs, tensor parallelism 2, four speculative draft tokens, 1,044 prompt tokens, and 512 generated tokens per request.

vLLM wins the single-request decode result and scales substantially better beyond four concurrent requests. SGLang wins at concurrency four and delivers lower TTFT through concurrency four.

• Single request: vLLM reached 109.36 tok/s, 12.12% faster than SGLang's 97.54 tok/s.
• Best SGLang point: 294.13 tok/s at concurrency 4, 10.36% faster than vLLM at the same concurrency.
• Best aggregate result: vLLM reached 430.67 tok/s at concurrency 16, 66.49% faster than SGLang.
• SGLang was capped to five live requests by its Mamba state cache. At offered concurrency 8 and 16, queued requests increased mean TTFT to 3.40 s and 10.01 s.

Test setup

Single-request measurements use two warmups and five measured iterations. Concurrency 2, 4, 8, and 16 use one warmup and three measured iterations. Every measured request used the same prompt and output-token limit.

Runtime configuration parity and necessary differences

Both engines used the same target, draft checkpoint, draft count, tensor parallelism, context limit, offered concurrency, request body, hardware, and client measurement path.

This is apples-to-apples at the model, hardware, request, and offered-load levels. It is not identical internally: each engine owns its cache allocator, graph implementation, scheduler, and kernels. Those differences are the subject of the comparison.

Primary results

All values below are arithmetic means across measured iterations or measured requests.

SGLang's throughput peaks at concurrency four. Its concurrency-eight and concurrency-sixteen runs offered the requested load, but the runtime admitted only five requests at once and queued the remainder. vLLM continued scaling through concurrency sixteen.

Decode-throughput distributions

tok/s is aggregate output throughput for the full concurrent request window.

Time-to-first-token distributions

TTFT counts individual requests, not iteration windows. This table exposes queueing that a throughput-only comparison would hide.

At concurrency eight, SGLang's p50 remains low because five requests start immediately while three wait for the next scheduler wave. The mean and maximum expose the queued tail. At concurrency sixteen, requests complete across four admission waves of 5 + 5 + 5 + 1.

Total-throughput distributions

tokSTotal combines prompt and output tokens over elapsed request time as reported by localmaxxing-cli. It is included for completeness; decode throughput and TTFT remain the more actionable interactive-serving metrics.

DFlash acceptance and runtime behavior

The logging windows and rate definitions differ between engines, so these values validate that DFlash was active but should not be treated as a precise cross-engine acceptance benchmark.

SGLang startup explicitly reported:

• DFlash2DraftModel initialized and ready.
• Fused DFlash KV materialization enabled.
• Decode CUDA graphs active.
• max_running_requests capped from 16 to 5 because the Mamba state cache allocated 29 slots and required five slots per request.

vLLM reported SpeculativeConfig(method='dflash', ...), DFlash speculative metrics, and sixteen simultaneously active requests during the matched high-concurrency run.

Why four draft tokens were used

A preliminary vLLM single-request sweep selected four drafts before the engine comparison.

A matched non-speculative vLLM run reached 65.28 tok/s, so four-token DFlash improved vLLM single-request decode throughput by 67.52%. A previously observed 96.6 tok/s endpoint was excluded as an autoregressive baseline after its recovered command showed MTP with two speculative tokens.

Reproduction commands

The common localmaxxing-cli workload was:

lmx speed-test run <vllm|sglang> \
  --mode remote \
  --base-url <endpoint> \
  --hf-id btbtyler09/Qwen3.8-27B-GPTQ-4bit \
  --served-model <served-alias> \
  --quantization GPTQ-4bit \
  --hardware hardware.json \
  --spec-method dflash \
  --spec-draft-model z-lab/Qwen3.8-27B-DFlash2 \
  --spec-num-tokens 4 \
  --prompt-tokens 1044 \
  --max-tokens 512 \
  --concurrency <1|2|4|8|16> \
  --out <payload.json>

The vLLM server used:

vllm serve /models/Qwen3.8-27B-GPTQ-4bit \
  --tensor-parallel-size 2 \
  --max-model-len 8192 \
  --max-num-seqs 16 \
  --gpu-memory-utilization 0.90 \
  --speculative-config \
  '{"method":"dflash","model":"/models/Qwen3.8-27B-DFlash2","num_speculative_tokens":4}'

The SGLang server used equivalent model and workload settings:

python -m sglang.launch_server \
  --model-path /models/Qwen3.8-27B-GPTQ-4bit \
  --tp-size 2 \
  --context-length 8192 \
  --mem-fraction-static 0.90 \
  --max-running-requests 16 \
  --disable-custom-all-reduce \
  --cuda-graph-backend-prefill disabled \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path /models/Qwen3.8-27B-DFlash2 \
  --speculative-num-draft-tokens 4

Conclusion

For this dual-RTX-3090 Qwen3.8-27B GPTQ-4bit deployment, vLLM is the stronger general serving choice. It wins single-request decode, remains competitive at concurrency two, and continues scaling to 430.67 tok/s at concurrency sixteen without the severe queueing seen in SGLang.

SGLang is compelling at low concurrency. It has lower TTFT at concurrency one through four and posts the best concurrency-four result: 294.13 tok/s with 104.27 ms mean TTFT. For a controlled deployment guaranteed to stay at four or fewer simultaneous requests, SGLang is a valid performance winner.

The crossover is operational, not cosmetic. SGLang's Mamba state-cache allocator capped this configuration to five active requests, while vLLM admitted sixteen. Once offered load exceeded that cap, SGLang's aggregate throughput fell and tail latency expanded to 8.9–26.6 seconds. On this hardware and software revision, choose SGLang for tightly bounded low-concurrency latency and vLLM for single-stream decode speed, load tolerance, and aggregate throughput.