The 4090's speed with half the VRAM story: 717 GB/s and 16 GB — the Ti Super's class at 320 W and the 4080's launch price, which is to say: the wrong side of that price line.
The 4080 is a 9728-core AD102 with a 256-bit bus at 717 GB/s — ~6.7% above the Ti Super, ~35% below the 4090. Token generation: ~80–105 t/s on 8B, ~48–62 t/s on 13B Q4, ~35–45 t/s on 13B Q6. Prompt processing (~950–1250 t/s on 13B) is the strongest in the 16 GB tier. It is, in pure performance terms, a great card. The problem is the receipt: it launched next to the 4070 Ti Super for the same VRAM, ~7% less bandwidth and a ~2× price.
Same model list as the Ti Super: 13B at Q6/IQ8, MoE 30B at Q3, 27B Q3 at short context. The 256-bit bus doesn't change what fits — it only changes how fast it streams, and here the streaming is genuinely good.
320 W, 750 W PSU, 3-slot Founders cooling. In a multi-card plan the 4080 is the 16 GB tile with the best per-tile bandwidth in its generation — a pair (32 GB, 640 W) is a proper 27B Q4/Q5 rig.
The MTP question: if your model ships a built-in multi-token-prediction head
(Qwen 3.5/3.6/3.8, Gemma 4), the one-flag speculative pass
(--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1) is worth
+45–70% (estimated) on token generation here; no per-card A/B in the record; the estimate sits between the measured 3090 Ti (+45%) and 4090 (+60%) results at n-max 2. The price is ~0.6–2 GB of VRAM, the
output is bit-identical to the unaccelerated run, and models without MTP heads (Llama class) see
nothing. Full tuning rules in the MTP guide.
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B Q8_0 | ~9.7 GB | ~80–105 t/s | ~116–179 t/s | ~950–1250 t/s | ✅ 32K+ ctx |
| 13B Q4_K_M | ~8.5 GB | ~48–62 t/s | ~69–105 t/s | ~950–1250 t/s | ✅ 32K+ ctx |
| 13B Q6_K | ~13.8 GB | ~35–45 t/s | ~50–77 t/s | ~800–1000 t/s | ✅ 8–16K ctx |
| 27B Q3_K_M | ~15.8 GB | ~12–16 t/s | ~17–27 t/s | ~550–700 t/s | ⚠️ 4–8K ctx |
| MoE 30B-A3B (Q3) | ~14.5 GB | ~34–45 t/s | ~49–77 t/s | ~850–1050 t/s | ⚠️ 4–8K ctx, 3B active |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Figures assume full offload (-ngl 99), ~8K context and batch 2048. Token speed tracks the memory bus almost linearly, so treat ranges as class estimates — your llama.cpp build and the model's attention layout shift them.
| Quant | GGUF size | 1× 4080 (16 GB) | 2× 4080 (32 GB) |
|---|---|---|---|
| Q3_K_M | ~15.8 GB | ⚠️ ~12–16 t/s, 4–8K ctx | ✅ (overkill) |
| Q4_K_M | ~19 GB | ❌ ~3 GB over | ✅ ~23–30 t/s, 8–16K ctx |
| Q5_K_M | ~22 GB | ❌ | ✅ ~20–25 t/s, 8K ctx |
| Q6_K | ~25 GB | ❌ | ✅ ~17–22 t/s, 8K ctx |
| Q8_0 | ~33 GB | ❌ | ❌ ~1 GB over |
Single-card 27B is the Q3 line, same as the Ti Super but ~10% faster. The pair (32 GB, 640 W) is the 4080's real story: 27B Q4 at 8–16K with the best split-rig bandwidth of the generation's 16 GB tier. A step below the 2× 3090 rig in everything, a step above the Ti Super pair in speed-per-tile.
-c 4096–8192 with quantized KV.PCIe 4.0, ~25–35% tax. 16 GB tiles: the pair lands on 27B Q4 at 32 GB with the fastest per-tile bus of the generation's 16 GB tier.
| Build | GPU power | VRAM | 13B Q8_0 | 27B Q4_K_M | 27B Q5_K_M |
|---|---|---|---|---|---|
| 2× 4080 | 640 W | 32 GB | ~58–74 t/s, 16K+ ctx | ✅ ~23–30 t/s, 8–16K ctx | ~20–25 t/s, 8K ctx |
The 2× 4080 rig is the closest Ada-built thing to a 3090 pair: similar VRAM (32 vs 48 GB), ~40% slower token speed (the split tax vs NVLink), at about half the build cost of a used 3090 pair.
When a model won't fit, -ngl N parks the tail layers on the CPU. The mental model
that saves pain: offloaded layers don't run at a fraction of GPU speed — they run at CPU
speed. Handy moves: peel the last 2–4 layers off to reclaim ~0.5–1 GB of VRAM for almost
nothing; keep MoE experts on CPU with --n-cpu-moe; park the output embedding with
-ot output=CPU on big models. Budget 32 GB system RAM for 27B-class offload, 64 GB for
70B. And never offload the KV cache — the session becomes unusable.
llama-server -m 27b-q4_k_m.gguf -ngl 99 -sm layer -c 16384 --tensor-split 1,1
llama-server -m llama-3.1-13b-q6_k.gguf -ngl 99 -c 16384 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The 4080 is a fast, well-specced card that lost a pricing war to its little brother. Performance: top of the 16 GB tier, no argument. Purchase: the 4070 Ti Super does 93% of the job at half the money. It re-enters the recommendation at a steep discount, in pairs, or as a used-market second tile for a 27B rig.