960 GB/s and 16 GB — the fastest 16 GB card ever made. 24 GB-class speed without the 24 GB.
The 5080's 256-bit GDDR7 bus (28 Gbps per pin) beats the 3090's 936 GB/s outright and comes within 5% of the 3090 Ti. Token generation: ~108–140 t/s on 8B, ~57–75 t/s on 13B Q4, ~43–57 t/s on 13B Q6. It is, per token, a 3090 Ti card — and prompt processing (~1000–1400 t/s on 13B) is the best in the tier, with 10752 cores.
Same VRAM as the 5070 Ti: 13B at Q6, MoE 30B at Q3, 27B at Q3 short-context. The 3090's 24 GB ran 27B Q4–Q5 single-card; the 5080 is faster and holds less. Everything 27B Q4+ is a 2-card project — and a 2× 5080 (32 GB) is the strongest 27B rig below the 5090: Q4 at ~40–52 t/s with 16K context.
360 W — between the 5070 Ti and the 3090, with Blackwell running a touch warmer than Ada at the same watts. 24/7 box: 850 W PSU for one card, 1000 W for two, and real case airflow.
Before the numbers: if your model ships a built-in multi-token-prediction head
(e.g. Qwen 3.5/3.6/3.8, Gemma 4), generation can be made faster with
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 — on this card that's worth
+45–75% (estimated) on token generation; no per-card community A/B exists for this card yet; the estimate follows the measured 3090/3090 Ti results at similar bandwidth. It costs ~0.6–2 GB extra VRAM and
does not change output quality, and it only applies to MTP-capable models — Llama-class
models are unaffected. Details and the tuning rules: MTP guide →
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B Q8_0 | ~9.7 GB | ~108–140 t/s | ~156–245 t/s | ~1000–1400 t/s | ✅ 32K+ ctx |
| 13B Q4_K_M | ~8.5 GB | ~57–75 t/s | ~82–131 t/s | ~1100–1400 t/s | ✅ 32K+ ctx |
| 13B Q6_K | ~13.8 GB | ~43–57 t/s | ~62–99 t/s | ~1000–1300 t/s | ✅ 8–16K ctx |
| 27B Q3_K_M | ~15.8 GB | ~18–25 t/s | ~26–44 t/s | ~800–1000 t/s | ⚠️ 4–8K ctx |
| MoE 30B-A3B (Q3) | ~14.5 GB | ~80–105 t/s | ~116–184 t/s | ~1000–1300 t/s | ⚠️ 4–8K ctx, 3B active |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Estimates for full GPU offload (-ngl 99), ~8K context, batch 2048. Token generation scales roughly with memory bandwidth; individual runs vary by model architecture (GQA vs MHA, MoE) and llama.cpp build. MoE token gen depends on active params, not total — treat as a bonus speed tier.
| Quant | GGUF size | 1× 5080 (16 GB) | 2× 5080 (32 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~3 GB over | ✅ ~40–52 t/s, 16K+ ctx |
| Q5_K_M | ~22 GB | ❌ | ✅ ~34–44 t/s, 8–16K ctx |
| Q6_K | ~25 GB | ❌ | ✅ ~30–38 t/s, 8–16K ctx |
| Q8_0 | ~33 GB | ❌ | ❌ ~1 GB over |
Single-card 27B is Q3. 2× (720 W, 32 GB) is the best 27B rig under the 5090: Q4_K_M at ~40–52 t/s with 16K+ context — faster than a 2× 3090 on the same model, at a larger split tax in exchange for NVLink. Q5 and Q6 fit at 8–16K; Q8 misses by the usual GB.
No NVLink: tensor split over PCIe 5.0, ~20–30% tax. 16 GB per tile: 2× reaches 32 GB.
| Build | GPU power | VRAM | 27B Q4_K_M | 27B Q5_K_M | 27B Q6_K |
|---|---|---|---|---|---|
| 2× 5080 | 720 W | 32 GB | ~40–52 t/s, 16K+ ctx | ~34–44 t/s, 8–16K ctx | ~30–38 t/s, 8–16K ctx |
The 2-card build is the card's endgame and the strongest 27B configuration below the 5090. Combined bandwidth (1920 GB/s) actually exceeds a 2× 3090 (1872 GB/s) — the PCIe 5.0 split tax is what keeps the 3090 pair competitive.
When a model doesn't fit, -ngl N keeps the last layers on the CPU instead.
The rule to internalize: offloaded layers run at CPU speed, not a percentage of GPU
speed. Practical patterns: drop just the last 2–4 layers to the CPU to reclaim ~0.5–1 GB
of VRAM (nearly free); keep MoE experts on CPU with --n-cpu-moe; park the output
embedding with -ot output=CPU to save another ~0.5–1 GB on big models. Budget 32 GB
of system RAM for 27B-class offload, 64 GB for 70B. Never offload the KV cache — the conversation
gets unusably slow.
llama-server -m 27b-q4_k_m.gguf -ngl 99 -sm layer -c 16384 --tensor-split 1,1
llama-server -m llama-3.1-13b-q6_k.gguf -ngl 99 -c 16384 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The speed king of the 16 GB tier — 3090 Ti-class token generation, the tier's best prompt processing, and a 2-card path to the best 27B rig under the flagship. Its one flaw is the same as the 5070 Ti's, amplified: 16 GB where a 3090 had 24. For 13B/MoE work it's simply fast; for single-card 27B Q4, it's the card that almost. The 5090 removes the cap entirely.