How Much VRAM to Run Llama 3.3 70B, Mistral, Gemma: The Complete 2026 Guide
Stop guessing. Here's how much VRAM you need for every major open LLM — with quantization estimates and third-party reference benchmark data, plus a free calculator to find your GPU match.
Estimate VRAM for your GPU & model →
Free interactive tool — no signup required
Why VRAM Is the Real Bottleneck
When running LLMs locally, VRAM (Video RAM) is the primary constraint.Every parameter, activation and attention head competes for that space, and unlike system RAM, GPU VRAM has no swap file.
But a model not fitting entirely in VRAM does not by itself prove that the model cannot execute. Some runtimes can place part of the weights, KV cache or other components in system RAM or other memory tiers. Execution feasibility and performance depend on the complete execution recipe — runtime, placement strategy, available RAM, context and other constraints. Offloading does not automatically make execution possible, and the outcome is not reducible to a single consequence.
So a 70B model at Q4 does not fit entirely in 8 GB of VRAM under this estimate. That is a statement about the VRAM estimate only — it is not proof that the model cannot execute. It is also why knowing exactly how much VRAM a model needs at each quantization level is the difference between hours of frustration and a working setup.
VRAM Requirements by Model (2026)
Below are the estimated VRAM requirements for the most popular open LLMs as of August 2026 — calculated from parameter counts and quantization size, including model weights + KV cache + activation memory. These are modelled values, not first-party measurements.
| Model | FP16 | Q8 | Q5 | Q4 | Q3 | Q2 |
|---|---|---|---|---|---|---|
| Llama 3.3 70B Instruct | 145 GB | 80 GB | 54 GB | 42 GB | 34 GB | 26 GB |
| Qwen 2.5 72B | 150 GB | 82 GB | 56 GB | 43 GB | 35 GB | 27 GB |
| Mistral Large 2 123B | 256 GB | 140 GB | 95 GB | 74 GB | 60 GB | 46 GB |
| Mistral 8x22B | 90 GB | 50 GB | 34 GB | 26 GB | 21 GB | 16 GB |
| Gemma 3 27B | 56 GB | 31 GB | 21 GB | 16 GB | 13 GB | 10 GB |
| DeepSeek V3 685B (package incl. MTP weights) | 1.4 TB | 770 GB | 520 GB | 405 GB | 330 GB | 250 GB |
| Llama 3.1 8B Instruct | 17 GB | 10 GB | 7 GB | 5.5 GB | 4.5 GB | 3.5 GB |
| Qwen 2.5 7B Instruct | 15 GB | 8.5 GB | 6 GB | 4.8 GB | 4 GB | 3 GB |
| Mistral Small 22B | 46 GB | 25 GB | 17 GB | 13 GB | 10.5 GB | 8 GB |
Values are calculated total VRAM estimates at context length 2048 tokens (prompt + generation). Longer contexts require more KV cache. Q4 = Q4_KM unless noted. Estimates modelled on llama.cpp-style memory layout (llama.cpp and ollama are common runtimes for these models).
Pro-tip: Context Window Is a VRAM Trap
The numbers above assume 2048 tokens of context. But VRAM is consumed by two things: the model weightsAND the KV cache — which grows linearly with context length. A 70B model at Q4 with a 32k context can eat +20 extra GBon top of the 42 GB base. At 128k tokens, you're looking at 60-80 GB just for the KV cache alone.
Rule: KV cache (GB) ≈ 2 × (model params in B) × (context in k-tokens) × (bytes per param)
Understanding Quantization Levels
Quantization reduces model size by storing weights with fewer bits. Here's what each level means:
16-bit float — Full precision. Used for training and fine-tuning. Quality: maximum.
2 bytes/param · Not recommended for inference on consumer GPUs
8-bit integer — Near-lossless. 4x size reduction. Quality: excellent.
~1 byte/param · Good balance of speed and quality for fine-tuning
5-bit — Very good quality. 3.2x reduction. Quality: very good.
~0.7 bytes/param · Hardly distinguishable from FP16 in most tasks
4-bit — The sweet spot. 4x reduction. Quality: very good for most tasks.
~0.5 bytes/param · Recommended for 70B models on consumer GPUs
3-bit — Aggressive compression. 5.3x reduction. Quality: good for casual use.
~0.4 bytes/param · Noticeable quality loss on complex reasoning
2-bit — Extreme compression. 6.6x reduction. Quality: noticeable loss.
~0.3 bytes/param · Use only when VRAM is critically limited
Quality Impact of Quantization
Reference quality scores for 4-bit quantized models across three tasks: factual recall, coding, and instruction following. Scores are relative to FP16 (100%). Reference values retained from the existing article — RigFact has not independently verified their provenance.
| Model | Quant | Factual | Coding | Instruct |
|---|---|---|---|---|
| Llama 3.3 70B | Q4_KM | 97% | 96% | 98% |
| Qwen 2.5 72B | Q4_KM | 96% | 97% | 97% |
| Mistral 8x22B | Q5 | 97% | 95% | 97% |
| Gemma 3 27B | Q4 | 95% | 94% | 96% |
| Llama 3.1 8B | Q4 | 94% | 93% | 95% |
Reference scores for MMLU (factual), HumanEval (coding), and MT-Bench (instruction following). Higher is better. Values retained from the existing article — provenance per row not independently verified by RigFact.
Which GPU for Which Model?
Based on the modelled estimates above, here is which GPU memory class each Q4 estimate falls into — an estimate, not a measurement. A memory fit is not proof of workload success — runtime, context and harness change the real result.
At the Q4 estimates in the table (2k context): Gemma 3 27B is 16 GB and sits inside 24 GB; Mistral 8x22B is 26 GB and is above it; Llama 3.1 8B at Q8 is a 10 GB estimate. All pre-2026 LEGACY models — estimates, not measurements.
Same 24 GB class — more headroom does not change the modelled values. Mistral Large 2 123B is a 46 GB estimate even at Q2, so it is above a 24 GB card. Overclocking adds compute, not VRAM capacity.
Professional card. Llama 3.3 70B at Q4 (42 GB) and Qwen 2.5 72B at Q4 (43 GB) sit inside a 48 GB estimate. Mistral Large 2 123B is 60 GB at Q3 — above this class. The 70B-class tier — all pre-2026 LEGACY models.
Datacenter GPU. Llama 3.3 70B at Q4 is a 42 GB estimate — slightly above 40 GB; at Q3 it is 34 GB. A100 80 GB gives the 70B class more headroom.
At the Q4 estimates in the table, everything through Mistral Large 2 123B (74 GB) sits inside 80 GB; DeepSeek V3 685B is 405 GB even at Q4. For the largest models, cloud GPU rental is the usual route — no hardware purchase.
Cloud GPU Alternatives
If buying a high-end workstation card isn't in the cards, cloud GPUs offer pay-per-minute access to much larger cards. Here are the main options in 2026:
We earn commissions when you shop through some of the links below at no additional cost to you.
Consumer GPUs (4090, A4000, A5000). Pricing and availability change; check the provider for current details.
Serverless and persistent endpoints for A100 40GB-class GPUs. Pricing and availability change; check the provider for current details.
Managed GPU droplets for A100-class GPUs, with simpler onboarding.
Note: For a 70B model like Llama 3.3 70B at Q4, the estimate lands around 40 GB (A100 40GB or equivalent). An estimate is not a fit determination — and sampled VRAM observed in a real run is not a minimum requirement either. Renting occasional cloud sessions avoids buying a second card; check providers for current pricing and availability.
Find Your VRAM Estimate
Use our free calculator to see which GPU you need for any model.
Open VRAM Calculator →Affiliate Disclosure
Some links on this page are affiliate links. If you purchase through them, we may earn a small commission at no additional cost to you. We only recommend tools we've thoroughly researched and believe add real value.
Our reviews and comparisons are based on objective analysis and are not influenced by affiliate partnerships.