Rent a GPU

Can the NVIDIA RTX 4090 run Gemma 3 4B?

RTX 4090 × Gemma 3 4B8K context
YesRuns fully on the GPU
Memory at FP1610.3 GB of 24 GB
Weights 8.6 GBKV cache 1.1 GBRuntime 0.6 GB
QuantFP16Full quality
Memory10.3 GBof 24 GB usable
Speed~70tokens per second
  1. FP16
  2. Q8_0
  3. Q6_K
  4. Q5_K_M
  5. Q4_K_M
  6. Q3_K_M
  7. Q2_K

Yes. The NVIDIA RTX 4090 can run Gemma 3 4B fully in VRAM at FP16 (full quality), using about 10.3 GB of the 24 GB available, at roughly 70 tokens/second.

Try other context lengths

Run it with Ollama

With Ollama installed, one command downloads and starts Gemma 3 4B:

ollama run gemma3:4b

Ollama's default download is Q4_K_M (about 4.3 GB here). You have room for FP16, which is full quality: pick that GGUF in LM Studio or from Hugging Face.

Run it in the cloud

No NVIDIA RTX 4090? Run Gemma 3 4B in the cloud.

Rent a cloud GPU by the hour. No upfront cost, stop any time.

Rent a GPU on RunPodRent a GPU on Vast.ai

Memory needed by quantization

At 8,192 tokens of context. The black line marks the 24 GB the NVIDIA RTX 4090 can use.

QuantQualityMemoryFits?Speed
FP16Full quality10.3 GBFits~70 tok/s
Q8_0Near-lossless6.3 GBFits~132 tok/s
Q6_KExcellent5.3 GBFits~172 tok/s
Q5_K_MVery good4.8 GBFits~198 tok/s
Q4_K_MGood, most popular4.3 GBFits~232 tok/s
Q3_K_MNoticeably degraded3.8 GBFits~288 tok/s
Q2_KHeavily degraded3.4 GBFits~375 tok/s

GPUs that run Gemma 3 4B comfortably

Models that run well on the NVIDIA RTX 4090

Frequently asked questions

Can NVIDIA RTX 4090 run Gemma 3 4B?

Yes. The NVIDIA RTX 4090 can run Gemma 3 4B fully in VRAM at FP16 (full quality), using about 10.3 GB of the 24 GB available, at roughly 70 tokens/second.

How much VRAM does Gemma 3 4B need?

About 4.3 GB at Q4_K_M with 8,192 tokens of context, and about 10.3 GB at full FP16 precision.

How fast is Gemma 3 4B on the NVIDIA RTX 4090?

Roughly 70 tokens per second at FP16, estimated from the NVIDIA RTX 4090's 1008 GB/s memory bandwidth.