GDDR6X and a 320-bit bus double the bandwidth of the mid-rangers. This is the first card where 13B models feel fast — the speed tier starts here.
Launch spec (19 Gbps GDDR6X). The 2022 refresh with 21 Gbps parts is effectively 832 GB/s — check your board; every number below scales up ~9% with it.
GDDR6X at 320 bits is a 2× bandwidth jump over the 256-bit 3070 class. Token generation follows: a 13B Q4_K_M that runs at ~20 t/s on a 3060 12GB runs at ~30–33 t/s here. That's the difference between "I'll check back in a minute" and "feels like a hosted API." For 8B models it's ~60 t/s — effectively unbounded for chat.
Ten GB is the odd number in the lineup, and it lands right in the awkward zone: 13B Q4_K_M (8.5 GB) fits, but with only ~1.5 GB for KV cache — expect a comfortable 4K–8K context, not 32K. A 13B at Q3 or an 8B at Q6 are the roomier fits. The 3080 Ti and 3060 12GB both solve this with 12 GB; the 3080 asks you to manage context length. If you want long-context 13B, look at the 3080 Ti or a 3090 instead.
8704 cores at ~1.71 GHz (plus GDDR6X's latency quirks) gives ~1200–1500 t/s prompt processing on 13B models. Long documents, big prompts, big batches: this is where the GA102's scale pays off, and where 2-GPU prompt throughput becomes genuinely practical.
Before the numbers: if your model ships a built-in multi-token-prediction head
(e.g. Qwen 3.5/3.6/3.8, Gemma 4), generation can be made faster with
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 — on this card that's worth
+40–60% (estimated) on token generation; the estimate follows the measured 3090 Ti number and the similar 320-bit bus. It costs ~0.6–2 GB extra VRAM and
does not change output quality, and it only applies to MTP-capable models — Llama-class
models are unaffected. Details and the tuning rules: MTP guide →
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B (Llama 3.1 8B, Mistral) | ~4.9 GB | ~77–104 t/s | ~108–166 t/s | ~1000–1200 t/s | ✅ big ctx, Q8_0 even |
| 13B Q4_K_M | ~8.5 GB | ~39–53 t/s | ~55–85 t/s | ~600–800 t/s | ✅ ~4K–8K ctx (tight) |
| 13B Q3_K_M | ~6.7 GB | ~50–64 t/s | ~70–102 t/s | ~700 t/s | ✅ with 16K ctx |
| 20B+ dense | ~12 GB | — | — | — | ❌ no |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Estimates, full GPU offload, launch-spec 760 GB/s. 21 Gbps refresh: add ~9%.
| Quant | GGUF size | 1× 3080 (10 GB) | 2× 3080 (20 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~9 GB over | ⚠️ ~18–20 t/s, tight: 4–8K ctx |
| Q5_K_M | ~22 GB | ❌ | ❌ ~2 GB over |
| Q6_K | ~25 GB | ❌ | ❌ |
| Q8_0 | ~33 GB | ❌ | ❌ |
The single card is the 13B specialist — 27B doesn't fit, period. The dual build is where the 3080 gets interesting: 20 GB holds a 27B Q4_K_M, and at ~18–20 t/s that's a proper chat speed. The catch is headroom: ~0.5–1 GB left for KV cache means you're living at 4K–8K context. It's the best 27B-Q4 build under the 24 GB cards — until the 3080 Ti pair adds 4 GB and ~40% speed for it.
-c 8192 is the config to tune; -c 16384 will OOM without KV-cache tuning (--cache-type-k q8_0 shrinks it ~2×).--cache-type-k/--cache-type-v q8_0) is the single best context-length trick on a 10 GB card.Single-card numbers above. GDDR6X bandwidth sums aggressively — 10 GB tiles with 760+ GB/s each — but no NVLink means every split pays the ~30% PCIe tax.
| Build | GPU power | VRAM | 13B Q4 | 27B Q4_K_M | 27B Q5_K_M | 27B Q6_K | 27B Q8_0 |
|---|---|---|---|---|---|---|---|
| 2× 3080 | 640 W | 20 GB | ~50 t/s | ⚠️ ~18–20 t/s, 4–8K ctx | ❌ ~2 GB over | ❌ | ❌ |
| 3× 3080 | 960 W | 30 GB | ~65 t/s | ~28–30 t/s | ~24–26 t/s | ⚠️ ~20–22 t/s, 8K ctx | ❌ |
| 4× 3080 | 1280 W | 40 GB | ~80–85 t/s | ~33 t/s | ~30–33 t/s | ~26–28 t/s | ⚠️ ~20 t/s, 8K ctx |
The 4-card build is the surprise: 40 GB reaches 27B Q8 without a 3090 in the box, and 3× comfortably runs Q5/Q6. The costs are honest — 1.4 kW of GPU, a TRX40/WRX80-class board with 3–4 x16/x8 slots, and 3-slot card spacing. As a bought-new build, a 2× 3090 (700 W, 48 GB, NVLink) outdoes the 3× and 4× 3080s on everything except 27B-Q8 context headroom.
When a model doesn't fit, -ngl N keeps the last layers on the CPU instead.
The rule to internalize: offloaded layers run at CPU speed, not a percentage of GPU
speed. A 27B model that's half offloaded is not 50% of GPU speed — it's ~3–5 t/s
total on a modern 8-core. Practical patterns: drop just the last 2–4 layers to the CPU to
reclaim ~0.5–1 GB of VRAM (nearly free); keep MoE experts on CPU with --n-cpu-moe;
park the output embedding with -ot output=CPU to save another ~0.5 GB on big
models. Budget 32 GB of system RAM for 27B-class offload, 64 GB for 70B. Never offload the
KV cache — the conversation gets unusably slow. The 20 GB pair is where offload earns its keep: 27B Q5 (22 GB) loads by parking the last ~8 layers on CPU at ~16–18 t/s. On 3×/4× you never need it.
llama-server -m 27b-q5_k_m.gguf -ngl 99 -sm layer -c 16384 --tensor-split 1,1,1llama-server -m 27b-q8_0.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1,1
llama-server -m llama-3-13b-q4_k_m.gguf -ngl 99 -c 8192 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The 3080 10GB is where the RTX 30 Series starts feeling like a "fast" LLM card: GDDR6X bandwidth turns 13B into a smooth experience, and 10 GB just about covers the standard model + quant combo. Its one weakness is context headroom. For 8–13B workloads at used prices it's a steal; for long-context or 20B+, step up to the 3080 Ti or the 3090.