Lottolabs

Model

unsloth/Laguna-S-2.1-GGUF

Display name

Laguna-S-2.1-GGUF

Base model

Laguna-S-2.1

Revision

main

Family

—

Parameters

118B

Active params

—

MoE

yes

Output

43.7 tok/s

Prefill

441 tok/s

Total

157 tok/s

TTFT

1,162 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

512

Output tokens

128

Prefill tokens

—

Context length

2048

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

IQ4_NL

Backend

cuda

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

/home/lotto/llama.cpp-laguna/build/bin/llama-bench -m /home/lotto/models/Laguna-S-2.1-GGUF/UD-IQ4_NL/Laguna-S-2.1-UD-IQ4_NL-00001-of-00003.gguf -fa on -sm layer -fitt 1024 -r 3 -o json

Extra flags

--sm layer --fitt 1024 --r 3 --o json

Notes

—

Reactions

—

Submitted

Aug 4, 2026, 3:43 PM

Last edited

—

Model

unsloth/Laguna-S-2.1-GGUF

Display name

Laguna-S-2.1-GGUF

Base model

Laguna-S-2.1

Revision

main

Family

—

Parameters

118B

Active params

—

MoE

yes

Output

43.4 tok/s

Prefill

535 tok/s

Total

62.7 tok/s

TTFT

118 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

63

Output tokens

128

Prefill tokens

—

Context length

2048

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

IQ4_NL

Backend

cuda

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

# Remote endpoint: http://127.0.0.1:8080 servedModel: /home/lotto/models/Laguna-S-2.1-GGUF/UD-IQ4_NL/Laguna-S-2.1-UD-IQ4_NL-00001-of-00003.gguf

Extra flags

—

Notes

—

Reactions

—

Submitted

Aug 4, 2026, 3:41 PM

Last edited

—

Model

unsloth/gemma-4-12b-it-GGUF

Display name

gemma-4-12b-it-GGUF

Base model

gemma-4-12B-it

Revision

main

Family

Gemma

Parameters

12B

Active params

—

MoE

no

Output

48.7 tok/s

Prefill

278 tok/s

Total

84.6 tok/s

TTFT

122 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

34

Output tokens

32

Prefill tokens

—

Context length

2048

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q8_K_XL

Backend

cuda

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

# Remote endpoint: http://127.0.0.1:8011 servedModel: gemma-4-12b-it

Extra flags

—

Notes

—

Reactions

—

Submitted

Jun 14, 2026, 9:52 PM

Last edited

—

Model

unsloth/Qwen3.5-9B-MTP-GGUF

Display name

Qwen3.5-9B-MTP-GGUF

Base model

Qwen3.5-9B

Revision

main

Family

Qwen

Parameters

9B

Active params

—

MoE

no

Output

70.6 tok/s

Prefill

4,425 tok/s

Total

—

TTFT

1,051 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

4650

Output tokens

64

Prefill tokens

4650

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

b1-8ad8aef

Quantization

Q8_K_XL

Backend

cuda

Tensor parallel

1

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.5-9B-UD-Q8_K_XL.gguf --ctx-size 262144 -ngl 999 -c 262144 --host 0.0.0.0 --port 8080

Extra flags

—

Notes

MTP-capable GGUF but MTP is disabled in llama.cpp launch (no --mtp / speculative decoding flags). Model emits reasoning_content only; content field is always empty. TTFT ≈ prefill_ms from llama.cpp timings. Decode speed measured at ~4.6K prompt tokens. KV cache stress tested up to 23.7K tokens with graceful ~25% decode degradation (70.6 → 54.6 tps).

Reactions

—

Submitted

Jun 4, 2026, 5:41 AM

Last edited

—

Model

unsloth/gemma-4-12b-it-GGUF

Display name

gemma-4-12b-it-GGUF

Base model

gemma-4-12B-it

Revision

main

Family

Gemma

Parameters

12B

Active params

—

MoE

no

Output

48.0 tok/s

Prefill

—

Total

40.9 tok/s

TTFT

1,035 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

100000

Output tokens

285

Prefill tokens

100000

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

b1-8ad8aef

Quantization

Q8_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-12b-it-UD-Q8_K_XL.gguf -ngl 999 -fa1 -c 0

Extra flags

--fa1

Notes

100K context tokens in KV cache. AMD Ryzen 9 9950X, 96GB DDR5. Gemma-4-12B-it (11.9B params). Q8_K_XL quant. Standard llama-server launch, no MTP. Prefill TPS not measured accurately.

Reactions

—

Submitted

Jun 4, 2026, 12:43 AM

Last edited

—

Model

unsloth/gemma-4-12b-it-GGUF

Display name

gemma-4-12b-it-GGUF

Base model

gemma-4-12B-it

Revision

main

Family

Gemma

Parameters

12B

Active params

—

MoE

no

Output

53.0 tok/s

Prefill

2,862 tok/s

Total

49.7 tok/s

TTFT

349 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

1000

Output tokens

284

Prefill tokens

—

Context length

131072

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

b1-8ad8aef

Quantization

Q8_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m gemma-4-12b-it-UD-Q8_K_XL.gguf -ngl 999 -fa1 -c 0

Extra flags

--fa1

Notes

AMD Ryzen 9 9950X, 96GB DDR5. Gemma-4-12B-it (11.9B params). Reasoning model with 262K vocab. Q8_K_XL quant. Standard llama-server launch, no MTP.

Reactions

—

Submitted

Jun 3, 2026, 10:45 PM

Last edited

—

Model

LiquidAI/LFM2.5-8B-A1B-GGUF

Display name

LFM2.5-8B-A1B-GGUF

Base model

LFM2.5-8B-A1B

Revision

main

Family

Llama

Parameters

8B

Active params

1B

MoE

yes

Output

227 tok/s

Prefill

—

Total

195 tok/s

TTFT

458 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

100000

Output tokens

512

Prefill tokens

100000

Context length

128000

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q8_0

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM2.5-8B-A1B-Q8_0.gguf -ngl 999 -fa1 -c 0

Extra flags

--fa1

Notes

100,000 tokens in KV cache (near max 128K context). 351 output tokens.

Reactions

—

Submitted

May 29, 2026, 6:14 AM

Last edited

—

Model

LiquidAI/LFM2.5-8B-A1B-GGUF

Display name

LFM2.5-8B-A1B-GGUF

Base model

LFM2.5-8B-A1B

Revision

main

Family

Llama

Parameters

8B

Active params

1B

MoE

yes

Output

259 tok/s

Prefill

—

Total

190 tok/s

TTFT

382 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

50000

Output tokens

512

Prefill tokens

50000

Context length

128000

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q8_0

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM2.5-8B-A1B-Q8_0.gguf -ngl 999 -fa1 -c 0

Extra flags

--fa1

Notes

50,000 tokens in KV cache. 269 output tokens.

Reactions

—

Submitted

May 29, 2026, 6:14 AM

Last edited

—

Model

LiquidAI/LFM2.5-8B-A1B-GGUF

Display name

LFM2.5-8B-A1B-GGUF

Base model

LFM2.5-8B-A1B

Revision

main

Family

Llama

Parameters

8B

Active params

1B

MoE

yes

Output

293 tok/s

Prefill

—

Total

256 tok/s

TTFT

251 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

10000

Output tokens

512

Prefill tokens

10000

Context length

128000

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q8_0

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM2.5-8B-A1B-Q8_0.gguf -ngl 999 -fa1 -c 0

Extra flags

--fa1

Notes

10,000 tokens in KV cache. 512 output tokens.

Reactions

—

Submitted

May 29, 2026, 6:14 AM

Last edited

—

Model

LiquidAI/LFM2.5-8B-A1B-GGUF

Display name

LFM2.5-8B-A1B-GGUF

Base model

LFM2.5-8B-A1B

Revision

main

Family

Llama

Parameters

8B

Active params

1B

MoE

yes

Output

303 tok/s

Prefill

—

Total

279 tok/s

TTFT

144 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

512

Prefill tokens

0

Context length

128000

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q8_0

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m LFM2.5-8B-A1B-Q8_0.gguf -ngl 999 -fa1 -c 0

Extra flags

--fa1

Notes

Cold start (no KV cache). 512 output tokens.

Reactions

—

Submitted

May 29, 2026, 6:14 AM

Last edited

—

Model

unsloth/Qwen3.6-27B-MTP-GGUF

Display name

Qwen3.6-27B-MTP-GGUF

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

48.0 tok/s

Prefill

—

Total

44.3 tok/s

TTFT

897 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

200000

Output tokens

512

Prefill tokens

200000

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q4_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

2

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-UD-Q4_K_XL.gguf -ngl 999 -fa1 -c 0 --spec-type draft-mtp --spec-draft-n-max 2

Extra flags

--fa1 --spec-type draft-mtp --spec-draft-n-max 2

Notes

KV cache stress test. 200,000 tokens in cache before generation. ~26% decode penalty vs cold start.

Reactions

—

Submitted

May 29, 2026, 5:49 AM

Last edited

—

Model

unsloth/Qwen3.6-27B-MTP-GGUF

Display name

Qwen3.6-27B-MTP-GGUF

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

59.3 tok/s

Prefill

—

Total

56.1 tok/s

TTFT

503 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

50000

Output tokens

512

Prefill tokens

50000

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q4_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

2

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-UD-Q4_K_XL.gguf -ngl 999 -fa1 -c 0 --spec-type draft-mtp --spec-draft-n-max 2

Extra flags

--fa1 --spec-type draft-mtp --spec-draft-n-max 2

Notes

KV cache stress test. 50,000 tokens in cache before generation.

Reactions

—

Submitted

May 29, 2026, 5:49 AM

Last edited

—

Model

unsloth/Qwen3.6-27B-MTP-GGUF

Display name

Qwen3.6-27B-MTP-GGUF

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

65.8 tok/s

Prefill

—

Total

63.2 tok/s

TTFT

320 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

10000

Output tokens

512

Prefill tokens

10000

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q4_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

2

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-UD-Q4_K_XL.gguf -ngl 999 -fa1 -c 0 --spec-type draft-mtp --spec-draft-n-max 2

Extra flags

--fa1 --spec-type draft-mtp --spec-draft-n-max 2

Notes

KV cache stress test. 10,000 tokens in cache before generation.

Reactions

—

Submitted

May 29, 2026, 5:49 AM

Last edited

—

Model

unsloth/Qwen3.6-27B-MTP-GGUF

Display name

Qwen3.6-27B-MTP-GGUF

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

65.2 tok/s

Prefill

—

Total

60.8 tok/s

TTFT

567 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

0

Output tokens

512

Prefill tokens

0

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q4_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

2

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-UD-Q4_K_XL.gguf -ngl 999 -fa1 -c 0 --spec-type draft-mtp --spec-draft-n-max 2

Extra flags

--fa1 --spec-type draft-mtp --spec-draft-n-max 2

Notes

KV cache stress test. 0 tokens in cache (cold start).

Reactions

—

Submitted

May 29, 2026, 5:48 AM

Last edited

—

Model

unsloth/Qwen3.6-27B-MTP-GGUF

Display name

Qwen3.6-27B-MTP-GGUF

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

64.9 tok/s

Prefill

603 tok/s

Total

61.2 tok/s

TTFT

497 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

300

Output tokens

544

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q4_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

2

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-UD-Q4_K_XL.gguf -ngl 999 -fa1 -c 0 --spec-type draft-mtp --spec-draft-n-max 2

Extra flags

--fa1 --spec-type draft-mtp --spec-draft-n-max 2

Notes

Unsloth Dynamic quantization. tokS includes reasoning_content + visible content. MTP speculative decoding (2 draft tokens). -fa1 flash attention. -c 0 = full 262K context.

Reactions

—

Submitted

May 29, 2026, 5:26 AM

Last edited

—

Model

unsloth/Qwen3.6-27B-MTP-GGUF

Display name

Qwen3.6-27B-MTP-GGUF

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

65.3 tok/s

Prefill

89.2 tok/s

Total

60.7 tok/s

TTFT

336 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

30

Output tokens

288

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q4_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

2

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-UD-Q4_K_XL.gguf -ngl 999 -fa1 -c 0 --spec-type draft-mtp --spec-draft-n-max 2

Extra flags

--fa1 --spec-type draft-mtp --spec-draft-n-max 2

Notes

Unsloth Dynamic quantization. tokS includes reasoning_content + visible content. MTP speculative decoding (2 draft tokens). -fa1 flash attention. -c 0 = full 262K context.

Reactions

—

Submitted

May 29, 2026, 5:25 AM

Last edited

—

Model

unsloth/Qwen3.6-27B-MTP-GGUF

Display name

Qwen3.6-27B-MTP-GGUF

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

70.8 tok/s

Prefill

2,677 tok/s

Total

69.1 tok/s

TTFT

374 ms

Peak VRAM

—

Power draw

—

Hardware cost

—

Prompt tokens

1000

Output tokens

1056

Prefill tokens

—

Context length

262144

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

llama.cpp

Engine version

—

Quantization

Q4_K_XL

Backend

—

Tensor parallel

—

Pipeline parallel

—

GPU layers

999

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

yes

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

yes

Spec method

draft-mtp

Spec model

—

Spec draft model

—

Spec tokens

2

Spec ngram

—

Spec draft TP

—

Spec draft window

—

MTP enabled

no

MTP draft layers

—

Temperature

—

Top P

—

Top K

—

Min P

—

Repeat penalty

—

Mirostat

—

Command

llama-server -m Qwen3.6-27B-UD-Q4_K_XL.gguf -ngl 999 -fa1 -c 0 --spec-type draft-mtp --spec-draft-n-max 2

Extra flags

--fa1 --spec-type draft-mtp --spec-draft-n-max 2

Notes

Unsloth Dynamic quantization. tokS includes reasoning_content + visible content. MTP speculative decoding (2 draft tokens). -fa1 flash attention. -c 0 = full 262K context.

Reactions

—

Submitted

May 29, 2026, 5:25 AM

Last edited

—

Model

unsloth/Qwen3.6-27B

Display name

Qwen3.6-27B

Base model

Qwen3.6-27B

Revision

main

Family

Qwen

Parameters

27B

Active params

—

MoE

no

Output

45.2 tok/s

Prefill

1,211 tok/s

Total

42.8 tok/s

TTFT

210 ms

Peak VRAM

24.3GB

Power draw

—

Hardware cost

—

Prompt tokens

512

Output tokens

256

Prefill tokens

—

Context length

8192

Batch size

1

Hardware class

DISCRETE_GPU

Hardware

RTX 3090 48GB ×2

GPU slots

—

GPU count

2

VRAM

48GB

Chip vendor

—

Chip family

—

Chip variant

—

Unified memory

—

NPU TOPS

—

CPU

AMD Ryzen

RAM

64GB

OS

Ubuntu

Power

—

Engine

vllm

Engine version

0.8.4

Quantization

Q4_K_M

Backend

cuda

Tensor parallel

—

Pipeline parallel

—

GPU layers

—

Split mode

—

KV cache dtype

—

KV cache size

—

Prefix caching

—

Attention backend

—

Flash attention

—

Chunked prefill

—

Prefill chunk

—

Continuous batching

—

CPU offload

—

CPU layers

—

Rope scaling

—

Rope scale

—

Yarn ext factor

—

Engine quant

—

SGLang quant

—

GPU mem util

—

Max running seqs

—

Scheduler delay

—

Num parallel

—

Concurrency

—

Spec decoding

no

Spec method

—

Spec model

—

Spec draft model

—

Spec tokens

—

Spec ngram

—