A 256-bit bus and 448 GB/s make it the fastest 8 GB card of its generation. Great card — right up against its 8 GB ceiling.
The 256-bit bus is where the GA104 earns its mid-range price. Token generation tracks bandwidth closely, so an 8B Q4 model runs at ~45 t/s versus ~35 on the 3060 — the difference between "fast" and "instant." Prompt processing scales too: ~800 t/s, which means long prompts and big system messages are digested before you finish typing the next.
More bandwidth can't help a model that doesn't fit. A 13B Q4_K_M (~8.5 GB + KV) still overflows, so this card lives in the same "excellent 8B / squeezed 13B-Q3" regime as the 3060 8GB — just 25% faster inside it. That's the key comparison on this site: a 3060 12GB and a 3070 cost similar money, and the 3060 12GB runs a bigger, higher-quality model. The 3070 only wins if the 8B class is all you'll ever run and speed is the metric.
5888 cores is plenty for prompt processing at this VRAM class. Nothing to complain about; the compute is never the constraint on this card.
Before the numbers: if your model ships a built-in multi-token-prediction head
(e.g. Qwen 3.5/3.6/3.8, Gemma 4), generation can be made faster with
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 — on this card that's worth
+40–60% (estimated) on token generation; no single-card 3070 A/B is in the community record yet. It costs ~0.6–2 GB extra VRAM and
does not change output quality, and it only applies to MTP-capable models — Llama-class
models are unaffected. Details and the tuning rules: MTP guide →
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B (Llama 3.1 8B, Mistral) | ~4.9 GB | ~59–77 t/s | ~83–123 t/s | ~700–900 t/s | ✅ comfortable, 16K+ ctx |
| 8B Q6_K | ~6.3 GB | ~49–64 t/s | ~69–102 t/s | ~650 t/s | ✅ with 8K ctx |
| 13B Q4_K_M | ~8.5 GB | — | — | — | ❌ just over budget |
| 13B Q3_K_M | ~6.7 GB | ~31–40 t/s | ~43–64 t/s | ~500 t/s | ⚠️ quality loss |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Estimates, full GPU offload, ~8K context.
| Quant | GGUF size | 1× 3070 (8 GB) | 2× 3070 (16 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~11 GB over | ❌ ~3 GB over |
| Q5_K_M | ~22 GB | ❌ | ❌ |
| Q6_K | ~25 GB | ❌ | ❌ |
| Q8_0 | ~33 GB | ❌ | ❌ |
The 448 GB/s bus is fast — but bandwidth can't move bytes that don't exist. 27B is out of reach single and doubled, exactly like the 3060 8GB. Partial CPU offload (≈14 layers on GPU) gets you ~4–6 t/s on Q4_K_M: faster than the 3060's offload thanks to the extra layers per watt of GPU, still slow by LLM standards. The honest 27B route from a 3070 owner is "add a 3090" (mixed 36 GB build) rather than a second 3070.
Single-card numbers above; below, the tiling ladder for the fastest 8 GB card of its generation. 448 GB/s per tile sums well, but there's no NVLink, so splits pay the ~30% PCIe tax.
| Build | GPU power | VRAM | 13B Q4 | 27B Q4_K_M | 27B Q5_K_M | 27B Q6/Q8 |
|---|---|---|---|---|---|---|
| 2× 3070 | 440 W | 16 GB | ~40 t/s | ❌ ~3 GB over | ❌ | ❌ |
| 3× 3070 | 660 W | 24 GB | ~38–42 t/s | ~15–17 t/s, 8K ctx | ~13 t/s | ❌ |
| 4× 3070 | 880 W | 32 GB | ~50 t/s | ~20–22 t/s | ~17–19 t/s | ❌ |
The number to notice: 3× 3070 (24 GB, 660 W) lands at ~16 t/s on 27B Q4 — a single 3090 (24 GB, 350 W) does the same for half the power with room for NVLink later. The 4-card build (32 GB) is a fine 27B Q4/Q5 box if the cards are free; as a purchase it loses to 2× 3060 12G at nearly every price point. Boards with 3–4 x16/x8 slots (X570/X670E, or workstation) and an 800–1000 W PSU are the gating items.
When a model doesn't fit, -ngl N keeps the last layers on the CPU instead.
The rule to internalize: offloaded layers run at CPU speed, not a percentage of GPU
speed. A 27B model that's half offloaded is not 50% of GPU speed — it's ~3–5 t/s
total on a modern 8-core. Practical patterns: drop just the last 2–4 layers to the CPU to
reclaim ~0.5–1 GB of VRAM (nearly free); keep MoE experts on CPU with --n-cpu-moe;
park the output embedding with -ot output=CPU to save another ~0.5 GB on big
models. Budget 32 GB of system RAM for 27B-class offload, 64 GB for 70B. Never offload the
KV cache — the conversation gets unusably slow. On the 16 GB pair, 27B doesn't load at all — offload can't close a 3 GB gap cheaply, so treat 2× 3070 as a 13B machine, not a slow 27B machine.
llama-server -m 27b-q4_k_m.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1llama-server -m 27b-q5_k_m.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1,1
llama-server -m llama-3.1-8b-q5_k_m.gguf -ngl 99 -c 16384 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The fastest 8 GB card of its generation — and 8 GB is all it gets. If you already own it, enjoy a snappy 8B server and Q3 13B experiments. If you're buying, the 3060 12GB gives you more capability for similar money and the 3080 10GB is an outright upgrade. The 3070 sits in the classic awkward middle: more speed than the 12GB 3060, less headroom.