Three series, side by side

Full specification sheet

Green rows: the site's most-recommended cards per tier. All cards are PCIe x16 (4.0 on the RTX 30/40 Series, 5.0 on the RTX 50 Series) and supported by current llama.cpp CUDA builds. RTX 30 Series card names link to full guides. † Not listed on NVIDIA’s official comparison page (fetched 2026-09-24) — specs on that card’s page are flagged as supplementary.

Memory bandwidth, all 28 cards

On the same model, token-generation speed tracks this list almost linearly. The RTX 5090 alone covers the entire previous two generations' spread in one step above them.

VRAM tiers — what each step buys you

Multi-GPU across the series

llama.cpp splits transformer layers across cards (-sm layer). Token generation speed ≈ (sum of bandwidths) minus an inter-GPU tax:

• 70B build: 2× 3090 (NVLink, 48 GB) is still the price/speed reference; 2× 4090 (48 GB, PCIe) is the new-money equivalent at a larger tax; 2× 5090 (64 GB) is the over-VRAM option with plenty of headroom for Q5/Q6.
• Budget builds: 2–4× 3060 12G remains the cheap route to 24–48 GB; 50-series equivalents are faster per card but more expensive per GB at launch pricing.
• Mixed builds work: a 3090 + 5090 pair (56 GB) is a legitimate 70B build; --tensor-split lets you weight layers toward the faster card. The PCIe tax applies to traffic between different cards regardless of generation.

Full mechanics, CPU offloading and exact flags: multi-GPU guide →. RTX 30 Series build-by-build scaling tables: 30 Series comparison →.

Cross-series buyer's summary

Per-series comparisons