Determining the Required VRAM for Local LLM Deployment
Determining whether a local LLM will function on your hardware is the primary concern. The answer hinges on the model's size, the level of quantization applied, and the length of the context. This guide provides a practical starting point for selecting the appropriate amount of VRAM.
How quantization affects VRAM
Quantization reduces the precision used to store model weights. Lower-bit quantization makes the model smaller and uses less VRAM, with some loss in quality.
| Quant | Bits per weight | Typical use |
|---|---|---|
| Q8_0 | 8 | Very high quality |
| Q6_K | ~6.6 | Very good quality |
| Q5_K_M | ~5.5 | Good quality and size |
| Q4_K_M | ~4.5 | Good balance of size and quality |
| Q3_K_M | ~3.5 | Lower VRAM, more quality loss |
Q4_K_M is a common choice when VRAM is limited. If you have more VRAM available, Q5 or Q6 lets you run the same model with less quantization.
Approximate VRAM by model size
These are rough estimates for the model weights. The actual amount of VRAM required is higher because the runtime, KV cache, and context also use memory.
| Model size | Q8_0 | Q6_K | Q4_K_M | Q3_K_M |
|---|---|---|---|---|
| 4B | ~5 GB | ~4 GB | ~3 GB | ~2.5 GB |
| 8B | ~9 GB | ~7 GB | ~5.5 GB | ~4.5 GB |
| 12B | ~13 GB | ~10 GB | ~8 GB | ~6.5 GB |
| 14B | ~16 GB | ~12 GB | ~9 GB | ~7.5 GB |
| 27B | ~30 GB | ~22 GB | ~17 GB | ~13 GB |
| 32B | ~36 GB | ~27 GB | ~20 GB | ~16 GB |
| 70B | ~80 GB | ~60 GB | ~42 GB | ~34 GB |
These are estimates, not hard limits. Different model architectures and quantization formats can change the actual size.
What different amounts of VRAM can run
| VRAM | Practical range | Current examples |
|---|---|---|
| 8 GB | Small models around 4B to 9B | Gemma 4 E4B, Qwen3.5 9B |
| 12 GB | Small to mid-sized models around 9B to 14B | Gemma 4 12B, Qwen3.5 9B |
| 16 GB | 12B to 27B with lower quantization | Gemma 4 26B-A4B, Qwen3.6 27B at Q4 |
| 24 GB | 27B to 35B at Q4 to Q6 | Qwen3.8 27B, Gemma 4 31B |
| 32 GB | 27B to 35B at higher quantization | Qwen3.8 27B, Gemma 4 31B |
| 48 GB | Large dense models at lower quantization | 70B-class models at Q3 to Q4 |
| 80 GB | Large dense models at higher quantization | 70B-class models at Q4 to Q6 |
These ranges are for models whose weights can fit on the GPU. Large MoE models are different: only some of their parameters are active for each token, but the model still has to store its full set of weights. A model with 100B or more total parameters therefore does not fit in a 100B-sized VRAM budget simply because it has fewer active parameters.
MoE models
Mixture-of-Experts models contain multiple groups of parameters called experts. Only some experts are used for each token, which can make inference more efficient than a dense model with the same total parameter count.
However, the inactive experts are still part of the model. Large MoE models can therefore require much more memory than their active parameter count suggests. Very large models may need multiple GPUs or system RAM offloading.
Context length also uses VRAM
The model weights are only part of the memory requirement. The KV cache grows as the context gets longer, so running the same model at 64K context can require substantially more VRAM than running it at 4K.
- Longer context requires more VRAM.
- KV-cache precision affects memory usage.
- Batch size and concurrent users also increase memory usage.
- Leave some VRAM available for the runtime instead of filling the GPU completely with model weights.
Practical tips
- Check the actual size of the quantized model you want to run.
- Do not use the model file size as the exact VRAM requirement. Leave room for the KV cache and runtime.
- If a model does not fit entirely in VRAM, part of it can be offloaded to system RAM, but inference will usually be slower.
- For long-context or agentic workloads, budget more VRAM than the model weights alone require.
- Multiple GPUs can split a model when one GPU does not have enough VRAM.
Run it on DaDesktop
You do not need to buy a GPU to run a local LLM. DaDesktop gives you a cloud desktop with the VRAM you need, so you can run the model directly without owning the hardware.
Pick the VRAM tier that fits your model, load it, and start using it. No setup, no hardware purchase, no driver issues. See available GPUs for the options.