The speed bump you can't use: 288 GB/s with an 8 GB lid. Faster than the 4060, but the models that fit didn't change — the 16 GB sibling is the card this one exists to shadow.
The AD106 moves the bus to 288 GB/s — a step over the 4060's 272 — and the 4352 cores make prompt processing the best in the 8 GB tier (~400–500 t/s on small models). Token generation: ~40–54 t/s on 8B, ~21–29 t/s on 13B Q4. All of it is inside an 8 GB envelope, which is the point: the 4060 Ti 8G is the fastest version of a small card.
Same 8 GB math as every card in this tier: 8B at full quality, 13B Q4 at short context, 27B never. The extra speed makes the 13B squeeze more tolerable, but the VRAM decision is identical to the 4060's.
NVIDIA sold the same AD106 with 16 GB for a modest premium. For LLM work that changes the model tier, not the speed tier — which is why, on this site, the 16 GB version gets the recommendation and this page exists mainly to help you avoid buying the wrong one.
The MTP question: if your model ships a built-in multi-token-prediction head
(Qwen 3.5/3.6/3.8, Gemma 4), the one-flag speculative pass
(--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1) is worth
+40–70% (estimated) on token generation here; no per-card A/B in the record; the estimate leans on the low-bandwidth end where n-max 2 gains are largest. The price is ~0.6–2 GB of VRAM, the
output is bit-identical to the unaccelerated run, and models without MTP heads (Llama class) see
nothing. Full tuning rules in the MTP guide.
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B (Llama 3.1 8B, Mistral) | ~4.9 GB | ~40–54 t/s | ~56–92 t/s | ~400–500 t/s | ✅ Q8_0, 16K+ ctx |
| 13B Q4_K_M | ~8.5 GB | ~21–29 t/s | ~30–49 t/s | ~350–450 t/s | ⚠️ tight, short ctx only |
| 13B Q3_K_M | ~6.7 GB | ~24–33 t/s | ~34–56 t/s | ~400 t/s | ✅ with quality loss |
| 27B | ~19 GB | — | — | — | ❌ no |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Figures assume full offload (-ngl 99), ~8K context and batch 2048. Token speed tracks the memory bus almost linearly, so treat ranges as class estimates — your llama.cpp build and the model's attention layout shift them.
| Quant | GGUF size | 1× 4060 Ti 8G (8 GB) | 2× 4060 Ti 8G (16 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~11 GB over | ❌ ~3 GB over |
| Q5_K_M | ~22 GB | ❌ | ❌ |
| Q6_K | ~25 GB | ❌ | ❌ |
| Q8_0 | ~33 GB | ❌ | ❌ |
Identical to the rest of the 8 GB tier: nothing 27B-class fits until 4× (32 GB), where Q4_K_M runs at roughly ~9–12 t/s at 4–8K context. Paired 8 GB cards are a 16 GB card with extra parts — the 16 GB siblings do the same job for less.
PCIe 4.0, ~25–35% split tax, 8 GB per tile — the 4060's arithmetic, slightly faster.
| Build | GPU power | VRAM | 13B Q4 | 27B Q4_K_M |
|---|---|---|---|---|
| 2× 4060 Ti 8G | 320 W | 16 GB | ~29–38 t/s | ❌ ~3 GB over |
| 4× 4060 Ti 8G | 640 W | 32 GB | ~50–65 t/s | ~9–12 t/s, 4–8K ctx |
The 4-way build reaches 27B Q4 at a usable-ish clip. It also costs 640 W and four slots to do what two 16 GB cards do at 320 W.
When a model won't fit, -ngl N parks the tail layers on the CPU. The mental model
that saves pain: offloaded layers don't run at a fraction of GPU speed — they run at CPU
speed. Handy moves: peel the last 2–4 layers off to reclaim ~0.5–1 GB of VRAM for almost
nothing; keep MoE experts on CPU with --n-cpu-moe; park the output embedding with
-ot output=CPU on big models. Budget 32 GB system RAM for 27B-class offload, 64 GB for
70B. And never offload the KV cache — the session becomes unusable.
llama-server -m 13b-q4_k_m.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1
llama-server -m llama-3.1-8b-q6_k.gguf -ngl 99 -c 8192 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
Fastest 8 GB Ada, and the series' most avoidable purchase: the 16 GB sibling costs a little more and runs meaningfully different models at the same speed. As a cheap fast 8B box it's good; as an LLM card, it's a footnotes entry.