The budget classic. A 128-bit bus at 256 GB/s — a step above the 3050, a step below the 12 GB sibling. 8B models run comfortably — if only the memory was 4 GB bigger.
The 8 GB 3060 uses the 128-bit bus variant of the GA106: 256 GB/s — a third faster than the 3050's 224 GB/s, but 114 GB/s behind the 192-bit 12 GB version. An 8B Q4_K_M model (~4.9 GB of weights + KV cache) streams through at a pace that lands you in the ~34–47 t/s range — comfortable for chat. If the 12 GB version is close in price, that extra bandwidth (and 4 GB) is the better buy.
The 8 GB ceiling is what separates this card from its 12 GB sibling. A 13B Q4_K_M model is ~8.5 GB before the KV cache, so it simply doesn't fit at normal context. Your realistic options: 8B models at Q5/Q6 with a big context, or a 13B at Q3_K_M / IQ3 with a short context — noticeable quality loss. If 13B+ is on your roadmap, buy the 12 GB version instead; the silicon (GA106, 3584 cores) is otherwise identical — only the bus and memory differ (the 12 GB version runs a 192-bit bus at 360 GB/s).
3584 cores handles prompt processing for 8B models at a comfortable ~500–600 t/s. You won't feel first-token latency in normal use. MoE models (which activate far fewer parameters per token) run especially well here — the bandwidth and the small active set combine nicely.
Before the numbers: if your model ships a built-in multi-token-prediction head
(e.g. Qwen 3.5/3.6/3.8, Gemma 4), generation can be made faster with
--spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 — on this card that's worth
+40–70% (estimated) on token generation; there's no single-card 3060 A/B published yet; the estimate follows the 3× 3060 12G data point (measured +42%). It costs ~0.6–2 GB extra VRAM and
does not change output quality, and it only applies to MTP-capable models — Llama-class
models are unaffected. Details and the tuning rules: MTP guide →
| Model | Size on GPU | Token gen | Token gen (MTP est.) | Prompt proc. | Fits? |
|---|---|---|---|---|---|
| 8B (Llama 3.1 8B, Mistral) | ~4.9 GB | ~33–46 t/s | ~46–77 t/s | ~500–600 t/s | ✅ comfortable, 16K+ ctx |
| 8B Q6_K | ~6.3 GB | ~28–38 t/s | ~40–65 t/s | ~450 t/s | ✅ with 8K ctx |
| 13B Q4_K_M | ~8.5 GB | — | — | — | ❌ just over budget |
| 13B Q3_K_M | ~6.7 GB | ~18–24 t/s | ~25–41 t/s | ~350 t/s | ⚠️ quality loss, ~8K ctx |
| MoE 7–8B active (e.g. Qwen3-30B-A3B) | ~18 GB (A3B) | — | — | — | ❌ full; smaller MoE OK |
MTP est. = rough prior with --spec-type draft-mtp for models that ship MTP heads (Qwen 3.5/3.6/3.8, Gemma 4); Llama-class models get no MTP speedup. Actual gains depend on model, quant, context length, llama.cpp build and your card — sweep --spec-draft-n-max, don't trust the column blindly. MTP guide →
Estimates, full GPU offload, ~8K context. MoE models scale with active parameters, not total — a 30B-A3B MoE generates like a ~3–5B dense model, which suits this card.
| Quant | GGUF size | 1× 3060 8G (8 GB) | 2× 3060 8G (16 GB) |
|---|---|---|---|
| Q4_K_M | ~19 GB | ❌ ~11 GB over | ❌ ~3 GB over |
| Q5_K_M | ~22 GB | ❌ | ❌ |
| Q6_K | ~25 GB | ❌ | ❌ |
| Q8_0 | ~33 GB | ❌ | ❌ |
Same dead end as the 3050: no 27B quant fits in 8 GB or 16 GB. Partial CPU offload on a single
card (≈14 of 64 layers on GPU, -ngl 14) squeezes out ~3–5 t/s on Q4_K_M,
and needs 32 GB of system RAM to be comfortable. Runnable, not enjoyable. The interesting 27B
build with this chip family is the 3060 12GB pair — 24 GB
total, ~11 t/s on 27B Q4.
Single-card numbers are in the table above; below is what tiling four 170 W chips buys. No NVLink here, so every split pays the ~30% PCIe 4.0 tax.
| Build | GPU power | VRAM | 13B Q4 | 27B Q4_K_M | 27B Q5_K_M | 27B Q6/Q8 |
|---|---|---|---|---|---|---|
| 2× 3060 8G | 340 W | 16 GB | ~30 t/s | ❌ ~3 GB over | ❌ | ❌ |
| 3× 3060 8G | 510 W | 24 GB | ~33 t/s | ⚠️ ~12–14 t/s, 4–8K ctx | ❌ | ❌ |
| 4× 3060 8G | 680 W | 32 GB | ~42 t/s | ~16–18 t/s, 8K ctx | ⚠️ ~14 t/s, 4–8K ctx | ❌ |
The 4-card build (32 GB) is a legitimate 27B Q4/Q5 box at hobbyist prices, but compare it to the 3060 12GB lineup: two 12GB cards reach the same 24 GB at the same speed for fewer cards, and four reach 48 GB (27B Q6/Q8, 70B Q4) instead of 32. The 8GB tiles only win if they're already in the drawer.
When a model doesn't fit, -ngl N keeps the last layers on the CPU instead.
The rule to internalize: offloaded layers run at CPU speed, not a percentage of GPU
speed. A 27B model that's half offloaded is not 50% of GPU speed — it's ~3–5 t/s
total on a modern 8-core. Practical patterns: drop just the last 2–4 layers to the CPU to
reclaim ~0.5–1 GB of VRAM (nearly free); keep MoE experts on CPU with --n-cpu-moe;
park the output embedding with -ot output=CPU to save another ~0.5 GB on big
models. Budget 32 GB of system RAM for 27B-class offload, 64 GB for 70B. Never offload the
KV cache — the conversation gets unusably slow. On 16 GB builds, 27B Q4 needs the last ~4 layers on CPU to load — expect the ~8–10 t/s figures above to drop a notch if you lean on that.
llama-server -m 27b-q4_k_m.gguf -ngl 99 -sm layer -c 8192 --tensor-split 1,1,1llama-server -m 27b-q5_k_m.gguf -ngl 99 -sm layer -c 4096 --tensor-split 1,1,1,1
llama-server -m llama-3.1-8b-q6_k.gguf -ngl 99 -c 8192 -b 2048 -ub 2048 --host 127.0.0.1 --port 8080
The RTX 3060 8GB delivers excellent 8B-model performance per dollar — fast, cool (170 W), and easy. But its ceiling is exactly where this site's other cards start: the 12 GB sibling, the 3070, and especially the used 3080 all run the same 8B models faster and open the 13B+ door. It wins on pure cost, full stop.