VRAM calculator
Choose your hardware and an open model to see the memory it needs, how fast it runs, and what to change if it doesn’t fit.
Hardware
RTX 509032.0 GB usable
Model
21 of 47 fitollama run qwen3.8:27bOllama GGUF MLX Hugging Face Apache 2.0
Quality
Fewer bits per weight means a smaller model. 4-bit is the usual pick. Below that, small models lose quality fast; big ones hold up better.Context
How much text the model sees at once: your messages, its replies, pasted documents. It needs memory on top of the model.Chats at onceEach conversation keeps its own context, so memory multiplies.
Running cost
Power at the wall while it works, at your rate. Cloud is the cheapest on-demand GPU that runs the same job, at RunPod’s on-demand prices as of Oct 21, 2026. Add the hardware’s MSRP to see when owning pays off; the PDF report carries it into a budget table.Hours a day
Power at home
$9.30/mo
425 W
Cloud
$58/mo
2× RTX 2000 Ada at $0.48/h
Result
- GPU memory
- 29.9/ 32.0 GB
- Context
- 208K
- Speed
- 67tok/s
- First reply
- 0.5s
- Model
- 15.7 GB
- Context
- 13.1 GB
- Overhead
- 1.1 GB
- OS
- 0.0 GB
- Free
- 2.1 GB
Suggested changes
- 6-bit weightsBetter answers, 120K context, 52 tok/s
How the math works
- The model
- parameters × bits ÷ 8
- Qwen3.8 27B at 4-bit: 27.8B × 4.85 ÷ 8 = 15.7 GB.
- Context
- tokens × layers × KV heads × head size × 2 × bytes
- Llama 3.1 8B: 128 KB per token, 16.0 GB at 128K. Newer models cache far less, so we read each one’s config.
- Overhead
- driver + scratch buffers
- About 0.6 to 1.5 GB.
- Speed
- bandwidth ÷ bytes read per token
- Mixture-of-experts models read only their active experts, so they run faster than their size suggests.
- First reply
- 2 × active weights × prompt tokens ÷ TFLOPS
- Reading the prompt is compute, not bandwidth. A Mac can write as fast as a GeForce card and still read a long document several times slower.
- Images and video
- steps × weights × tokens × 2 ÷ TFLOPS
- Memory is the model plus one block’s working set. Time grows faster than resolution, since attention compares every patch with every other.
- Running cost
- watts × hours × your rate
- Set against the cheapest on-demand cloud GPU that runs the same job.
How much memory each model needs
Weights at each level, plus 32K tokens of context.
| Model | Parameters | 4-bit | 8-bit | As released | + 32K context |
|---|---|---|---|---|---|
| Nemotron 3 Nano 4B | 4B | 2.2 GB | 3.9 GB | BF167.4 GB | 592 MB |
| Qwen3.5 4B | 4.7B | 2.6 GB | 4.6 GB | BF168.7 GB | 1.0 GB |
| Gemma 4 E2B | 5.1B (2.3B active) | 2.9 GB | 5.1 GB | BF169.5 GB | 198 MB |
| Gemma 4 E4B | 8B (4.5B active) | 4.5 GB | 7.9 GB | BF1614.9 GB | 532 MB |
| Llama 3.1 8B | 8B | 4.5 GB | 7.9 GB | BF1615.0 GB | 4.0 GB |
| Ministral 3 8B | 8.9B | 5.0 GB | 8.8 GB | FP89.7 GB | 4.3 GB |
| Qwen3.5 9B | 9.7B | 5.4 GB | 9.5 GB | BF1618.0 GB | 1.0 GB |
| Gemma 4 12B | 12B | 6.8 GB | 11.8 GB | BF1622.3 GB | 576 MB |
| Ministral 3 14B | 13.9B | 7.9 GB | 13.8 GB | FP814.6 GB | 5.0 GB |
| gpt-oss 20B | 20.9B (3.6B active) | 11.8 GB | n/a | MXFP411.3 GB | 771 MB |
| LFM2.5 24B A2B | 24B (2B active) | 13.5 GB | 23.7 GB | BF1644.6 GB | 640 MB |
| Mistral Small 3.2 24B | 24B | 13.6 GB | 23.8 GB | BF1644.7 GB | 5.0 GB |
| Devstral Small 2 24B | 24B | 13.6 GB | 23.8 GB | FP824.0 GB | 5.0 GB |
| Gemma 4 26B A4B | 25.8B (3.8B active) | 14.6 GB | 25.5 GB | BF1648.1 GB | 520 MB |
| Qwen3.8 27B | 27.8B | 15.7 GB | 27.5 GB | BF1651.7 GB | 2.1 GB |
| Muse Glimmer 30B | 29.8B | 16.8 GB | 29.5 GB | BF1655.5 GB | 494 MB |
| GLM-4.7 Flash | 31.2B (3B active) | 17.6 GB | 30.9 GB | BF1658.2 GB | 1.7 GB |
| Gemma 4 31B | 31.3B | 17.7 GB | 30.9 GB | BF1658.2 GB | 2.0 GB |
| Nemotron 3.5 Lightning 30B A3B | 31.6B (3B active) | 17.8 GB | 31.2 GB | BF1658.8 GB | 239 MB |
| Olmo 3.1 32B | 32.2B | 18.2 GB | 31.9 GB | BF1660.0 GB | 2.8 GB |
| Qwen3.6 35B A3B | 36B (3B active) | 20.3 GB | 35.6 GB | BF1667.0 GB | 701 MB |
| Llama 3.3 70B | 70.5B | 39.8 GB | 69.8 GB | BF16131 GB | 10.0 GB |
| Nemotron 3.5 Bolt 74B A9B | 74.9B (9B active) | 42.3 GB | 74.1 GB | BF16139 GB | 418 MB |
| Qwen3 Coder Next 80B | 79.7B (3B active) | 45.0 GB | 78.8 GB | BF16148 GB | 842 MB |
| gpt-oss 120B | 117B (5.1B active) | 66.0 GB | n/a | MXFP459.0 GB | 1.1 GB |
| Mistral Small 4 119B | 119B (6.5B active) | 67.4 GB | n/a | FP8113 GB | 720 MB |
| Nemotron 3 Super 120B A12B | 124B (12B active) | 69.8 GB | 122 GB | BF16230 GB | 418 MB |
| Qwen3.5 122B A10B | 125B (10B active) | 70.6 GB | 124 GB | BF16233 GB | 915 MB |
| Mistral Medium 3.5 128B | 128B | 72.1 GB | n/a | FP8124 GB | 11.0 GB |
| Qwen3.8 Flash-Next | 180B (6B active) | 102 GB | 178 GB | BF16335 GB | 878 MB |
| MiniMax M2.7 | 229B (10B active) | 129 GB | n/a | FP8214 GB | 7.8 GB |
| DeepSeek V4 Flash | 284B (13B active) | 160 GB | n/a | FP4 + FP8155 GB | 199 MB |
| Hy3 | 295B (21B active) | 167 GB | 292 GB | BF16549 GB | 10.0 GB |
| MiMo V2.6 Flash | 309B (15B active) | 174 GB | n/a | FP4 + FP8161 GB | 744 MB |
| GLM-5.3 Flash | 321B (18B active) | 181 GB | n/a | FP8306 GB | 537 MB |
| Qwen3.5 397B A17B | 403B (17B active) | 228 GB | 399 GB | BF16751 GB | 1.1 GB |
| MiniMax M3 | 427B (23B active) | 241 GB | 423 GB | BF16795 GB | 3.8 GB |
| Nemotron 3 Ultra 550B A55B | 561B (55B active) | 316 GB | 555 GB | BF161,044 GB | 773 MB |
| DeepSeek V3.2 | 671B (37B active) | 379 GB | n/a | FP8642 GB | 2.4 GB |
| GLM-5.3 | 753B (40B active) | 425 GB | n/a | FP8704 GB | 3.0 GB |
| DeepSeek V4.1 Flash | 763B (16B active) | 431 GB | n/a | FP4 + FP8475 GB | 89 MB |
| Muse Spark Open 816B A42B | 822B (42B active) | 464 GB | 813 GB | BF161,531 GB | 2.4 GB |
| MiMo V2.6 Pro | 1T (42B active) | 578 GB | n/a | FP4 + FP8527 GB | 1.6 GB |
| Kimi K2.6 | 1T (32B active) | 580 GB | n/a | INT4554 GB | 2.1 GB |
| DeepSeek V4 Pro | 1.6T (49B active) | 903 GB | n/a | FP4 + FP8805 GB | 285 MB |
| Qwen3.8 2.4T A95B | 2.4T (95B active) | 1,381 GB | 2,421 GB | BF164,556 GB | 3.4 GB |
| Kimi K3 | 2.8T (104B active) | 1,570 GB | n/a | INT41,454 GB | 1.3 GB |
Image and video models
Questions people ask
How much VRAM do I need to run an LLM?
About 0.6 GB per billion parameters at 4-bit, plus context. An 8B model fits an 8 GB card, 27B to 32B wants 24 GB, and a dense 70B needs about 48 GB.
Which quantization should I use?
4-bit (Q4_K_M) unless you have room to spare. Below 4-bit, small models degrade fast; models over 100B hold up.
How much context do I need?
8K covers a normal chat, 32K a long one, 128K about a novel. Coding agents want 64K or more.
Why can my Mac only use part of its memory?
macOS lets the GPU use about 75% of memory. Raise it with sudo sysctl iogpu.wired_limit_mb=<MB>; it resets on reboot.
Can I run a model bigger than my GPU?
Yes, by spilling part of it into system RAM. Dense models slow down badly; mixture-of-experts models hold up.
Do two GPUs double my memory?
For fitting a model, yes. For speed, mostly no: with the default layer split the cards take turns.
Why does a Mac take longer to start answering?
Writing tokens depends on memory bandwidth, where Macs are strong. Reading the prompt depends on raw compute, where NVIDIA cards are several times faster, so long documents and coding agents wait longer on a Mac.
Ollama, LM Studio or vLLM?
Ollama or LM Studio for one person: easy, and they can spill into system RAM. vLLM for a team: it serves many chats at once but needs the whole model on the GPU.
How much VRAM do image and video models need?
FLUX.1 dev wants about 14 GB at FP8 and runs on a 12 GB card as a 6-bit GGUF. 14B video models want about 21 GB at FP8, and a 5-second 720p clip takes around half an hour on an RTX 4090. Distilled models like LTX-2.5 take about a minute.
How accurate is this?
Memory lands within about a gigabyte. Speed is a rough guide, plus or minus 25%.
Where does the model data come from?
Each model’s config.json on Hugging Face, with active-parameter counts from the model cards.
Need a machine that runs it?
Tell us the model and how many people use it. We’ll quote the hardware.