Rent a GPU

Nemotron 3 Nano 30B-A3B hardware requirements

31.6B parameters (3.5B active, mixture of experts). Needs about 19.8 GB at Q4_K_M with 8K context.

Parameters32B3.5B active (MoE)
Q4_K_M · 8K19.8 GBthe usual download
FP16 · 8K63.9 GBfull precision
Runs well on18/51GPUs and Macs we track

Memory needed at each quantization

Weights, KV cache and runtime overhead, at 8K and 32K tokens of context.

QuantQuality8K context32K context
FP16Full quality63.9 GB64 GB
Q8_0Near-lossless34.2 GB34.4 GB
Q6_KExcellent26.6 GB26.7 GB
Q5_K_MVery good23.1 GB23.3 GB
Q4_K_MGood, most popular19.8 GB20 GB
Q3_K_MNoticeably degraded16.1 GB16.2 GB
Q2_KHeavily degraded12.5 GB12.7 GB

Run it with Ollama

Downloads Q4_K_M by default.

ollama run nemotron-3-nano:30b

Model card

nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 on Hugging Face: licence, prompt format and GGUF builds.

Run it in the cloud

Run Nemotron 3 Nano 30B-A3B on a rented GPU today.

It needs about 19.8 GB at Q4_K_M: a rented RTX 3090 / RTX 4090 runs it for about $0.18 to $0.89 an hour.

RunPodRent a GPU on RunPodVast.aiRent a GPU on Vast.ai

Every GPU and Mac checked

Sorted by memory. Bars show how much of each card Nemotron 3 Nano 30B-A3B fills at the best quant that fits.

18 Runs well

NameMemory usedBest quantSpeed

15 Fits with heavy quantization

NameMemory usedBest quantSpeed

18 Too big

NameMemory usedBest quantSpeed