Skip to content

What can the RTX 4090 run?

What fits on the RTX 4090
Text models: 258Visual models: 17Strongest first
Memory
24 GB
Bandwidth
1,008 GB/s
Maker
NVIDIA

Open the calculator on this machineEvery model for 24 GB of VRAM

Text models that run on it

Text models that run on the RTX 4090, best first
ModelBest fitContextSpeedStrongest at
Qwen3.8 27B5-bit48K35 tok/sChat, Coding, Agents, Docs
Muse Glimmer 30B5-bit128K33 tok/sDocs
Qwen3.6 35B A3B4-bit72K197 tok/sChat, Coding, Agents, Docs
Gemma 4 31B4-bit80K35 tok/sChat, Coding, Agents, Docs
Gemma 4 26B A4B6-bit184K161 tok/sChat, Coding, Agents, Docs
Qwen3.5 9B8-bit256K71 tok/sChat, Coding, Agents, Docs
GLM-4.7 Flash5-bit24K171 tok/sChat, Coding, Agents
Gemma 4 12B8-bit256K51 tok/sChat, Coding, Docs
Nemotron 3.5 Lightning 30B A3B5-bit176K161 tok/sAgents, Docs
Qwen3.5 4B8-bit256K124 tok/sAgents, Docs
Devstral Small 2 24B6-bit20K35 tok/sCoding
LFM2.5 24B A2B6-bit128K212 tok/sAgents
Mistral Small 3.2 24B6-bit20K35 tok/sChat, Coding, Agents
Ministral 3 14B8-bit48K47 tok/sCoding, Agents, Docs
Gemma 4 E4B8-bit128K110 tok/sChat, Coding, Docs
gpt-oss 20BMXFP4128K197 tok/sAgents
Ministral 3 8B8-bit96K73 tok/sDocs
Gemma 4 E2B8-bit128K191 tok/sChat
Olmo 3.1 32B4-bit48K34 tok/sChat, Coding, Agents, Docs
Llama 3.1 8B8-bit112K82 tok/sChat, Coding, Agents, Docs
Qwen3 14B8-bit40K45 tok/sNot rated
Qwen3 VL 8B Instruct8-bit88K75 tok/sText and images
Qwen3 8B8-bit40K80 tok/sNot rated
Qwen2.5 7B Instruct8-bit32K87 tok/sNot rated
Qwen2.5 VL 7B Instruct8-bit125K80 tok/sText and images
Qwen3 VL 4B Instruct8-bit120K129 tok/sText and images
Qwen3 1.7B8-bit40K256 tok/sNot rated
Qwen3 4B8-bit40K140 tok/sNot rated
Qwen3 32B3-bit24K42 tok/sNot rated
Qwen3 4B Instruct 25078-bit128K140 tok/sNot rated
Qwen3.6 27B5-bit48K35 tok/sText and images
Qwen2.5 Coder 14B Instruct8-bit32K44 tok/sCoding
Qwen3 Coder 30B A3B Instruct5-bit16K169 tok/sCoding
Qwen3.5 27B5-bit48K35 tok/sText and images
Qwen3.5 35B A3B4-bit72K197 tok/sText and images
NVIDIA Nemotron 3 Nano 30B A3B BF165-bit176K161 tok/sNot rated
Qwen2.5 Coder 7B Instruct8-bit32K87 tok/sCoding
Mistral 7B Instruct V0.28-bit32K86 tok/sNot rated
Chandra OCR 28-bit256K111 tok/sReading documents
Qwen3 30B A3B5-bit16K169 tok/sNot rated
Qwen3 VL 2B Instruct8-bit176K247 tok/sText and images
Gemma 3 4B IT8-bit128K154 tok/sText and images
Qwen3 30B A3B Instruct 25075-bit16K169 tok/sNot rated
Llama 3.2 3B Instruct8-bit128K176 tok/sNot rated
Laguna XS.24-bit80K200 tok/sNot rated
OTel 2.0 LLM 31B IT4-bit80K35 tok/sNot rated
Qwen2.5 32B Instruct3-bit24K42 tok/sNot rated
MiniCPM V 4.7 35B A3B4-bit88K197 tok/sText and images
Qwen2.5 14B Instruct8-bit32K44 tok/sNot rated
Laguna XS 2.14-bit80K200 tok/sNot rated
Qwen3 1.7B Base8-bit32K291 tok/sNot rated
Diffusiongemma 26B A4B IT6-bit88K155 tok/sText and images
Qwen2.5 VL 32B Instruct3-bit24K41 tok/sText and images
Qwen2.5 Coder 32B Instruct3-bit24K42 tok/sCoding
Qwen2 VL 7B Instruct8-bit32K80 tok/sText and images
Ornith 1.5 9B8-bit256K71 tok/sNot rated
Granite 4.1 3B8-bit128K158 tok/sNot rated
Qwen3 VL 30B A3B Instruct4-bit44K183 tok/sText and images
Mistral 7B V0.18-bit32K86 tok/sNot rated
Meta Llama 3 8B Instruct Abliterated V38-bit8K82 tok/sNot rated

Best fit is the highest quality, up to 8-bit, that leaves the use case’s context. Ranked by our use-case scores, then by Hugging Face downloads. Speeds are rough, plus or minus 25%. The top 60 of 258 are shown.

Image and video models

Image and video models that run on the RTX 4090
ModelBest fitPer outputStrongest at
LTX-2.58-bit54 sVideo
Z-Image Turbo16-bit6.4 sImages
FLUX.1 [dev]16-bit12 sImages
Qwen-Image8-bit3.5 minImages
FLUX.2 [dev]5-bit1.8 minImages
Qwen-Image 2.116-bit1.5 minImages
FLUX.3 Klein16-bit8.9 sImages
Wan2.2 I2V A14B8-bit31 minVideo
FLUX.2 [klein] 9B16-bit3.5 sImages
Wan2.1 I2V 14B8-bit31 minVideo
Wan 2.1 T2V-14B8-bit39 minVideo
Krea 2 Turbo8-bit12 sImages
Stable Diffusion 3.5 Large16-bit48 sImages
Qwen Image 25128-bit3.5 minImages
Wan 2.2 TI2V-5B16-bit8.6 minVideo
Wan 2.2 A14B8-bit31 minVideo
HunyuanVideo 1.516-bit38 minVideo

Only just too big

These need up to a quarter more memory than the RTX 4090 has. Spilling into system RAM or a lower precision can get them running, slowly.

Questions

What is the best LLM for the RTX 4090?

Of the open models we track, the strongest that run well on the RTX 4090 are Qwen3.8 27B, Muse Glimmer 30B, Qwen3.6 35B A3B. 258 text models fit it in all.

Calculated from published specs, not benchmarked. All machines