The refresh that mostly helps first tokens: 7168 cores on the same 504 GB/s bus — prompt processing up ~20%, token speed and VRAM unchanged.
The Super is the AD103 with 22% more CUDA cores (7168 vs 5888) — and the same 192-bit GDDR6X bus. Since the two inference phases have different bottlenecks, the refresh splits cleanly:
For chat — which is mostly token streaming — the Super feels like the 4070. For prompt-heavy work (RAG, agentic loops, long-document QA) it earns the delta.
12 GB is 12 GB: 13B Q4 at 32K+ is the comfort zone, 27B is 7 GB over. Nothing about the refresh moves that line, which is the part LLM users care about most.
The MTP question: if your model ships a built-in multi-token-prediction head
(Qwen 3.5/3.6/3.8, Gemma 4), the one-flag speculative pass
(--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1) is worth
+40–65% (estimated) on token generation here; no per-card A/B in the record; the estimate matches the 4070's bus class at n-max 2. The price is ~0.6–2 GB of VRAM, the
output is bit-identical to the unaccelerated run, and models without MTP heads (Llama class) see
nothing. Full tuning rules in the MTP guide.
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B Q8_0 | ~9.7 GB | ~55–72 t/s | ~77–119 t/s | ~700–900 t/s | ✅ 32K ctx |
| 13B Q4_K_M | ~8.5 GB | ~35–46 t/s | ~49–76 t/s | ~750–950 t/s | ✅ 32K+ ctx |
| 13B Q5_K_M | ~11.9 GB | ~25–33 t/s | ~35–54 t/s | ~600–800 t/s | ⚠️ 8K ctx |
| 27B Q4_K_M | ~19 GB | — | — | — | ❌ ~7 GB over |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Figures assume full offload (-ngl 99), ~8K context and batch 2048. Token speed tracks the memory bus almost linearly, so treat ranges as class estimates — your llama.cpp build and the model's attention layout shift them. Token gen matches the RTX 4070 exactly; prompt proc. is ~20% higher.
| Quant | GGUF size | 1× 4070 S (12 GB) | 2× 4070 S (24 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~7 GB over | ⚠️ ~16–20 t/s, 8K ctx |
| Q5_K_M | ~22 GB | ❌ | ❌ ~2 GB over |
| Q6_K | ~25 GB | ❌ | ❌ |
| Q8_0 | ~33 GB | ❌ | ❌ |
Mirror image of the 4070: 27B Q4 at 2× (24 GB, 8K, unhurried), Q5 at 3× (36 GB, 8–16K). Multi-card token speeds match the 4070 — the Super's extra cores only help the prefill side of those builds.
PCIe 4.0, ~25–35% tax, 12 GB tiles — the 4070's arithmetic, prompt side faster.
| Build | GPU power | VRAM | 13B Q8 | 27B Q4_K_M | 27B Q5_K_M |
|---|---|---|---|---|---|
| 2× 4070 S | 440 W | 24 GB | ~40–52 t/s | ⚠️ ~16–20 t/s, 8K ctx | ❌ ~2 GB over |
| 3× 4070 S | 660 W | 36 GB | (overkill) | ~19–24 t/s, 16K+ ctx | ~14–18 t/s, 8–16K ctx |
3× (36 GB) is the interesting configuration — 27B Q5 with a real context. Board requirement as the 4070: three working x8/x16 lanes.
When a model won't fit, -ngl N parks the tail layers on the CPU. The mental model
that saves pain: offloaded layers don't run at a fraction of GPU speed — they run at CPU
speed. Handy moves: peel the last 2–4 layers off to reclaim ~0.5–1 GB of VRAM for almost
nothing; keep MoE experts on CPU with --n-cpu-moe; park the output embedding with
-ot output=CPU on big models. Budget 32 GB system RAM for 27B-class offload, 64 GB for
70B. And never offload the KV cache — the session becomes unusable.
llama-server -m 27b-q5_k_m.gguf -ngl 99 -sm layer -c 16384 --tensor-split 1,1,1
llama-server -m llama-3.1-13b-q4_k_m.gguf -ngl 99 -c 32768 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The 4070 with a ~20% faster prefill and nothing else moved. As a purchase: same price, take it; premium price, skip it. As a platform: the 4070's verdict applies wholesale — a 13B card with a 12 GB ceiling and a multi-card road to 27B.