Rent a GPU

Can the NVIDIA RTX 4060 run Llama 4 Scout 17B-16E?

RTX 4060 × Llama 4 Scout 17B-16E8K context
NoDoes not fit in memory
Memory at Q4_K_M68.3 GB of 8 GB
Weights 66.1 GBKV cache 1.6 GBRuntime 0.6 GB
  1. FP16
  2. Q8_0
  3. Q6_K
  4. Q5_K_M
  5. Q4_K_M
  6. Q3_K_M
  7. Q2_K

No. Even at Q4_K_M, Llama 4 Scout 17B-16E needs about 68.3 GB, but the NVIDIA RTX 4060 has 8 GB usable. You could offload some layers to system RAM, but it will be very slow.

Try other context lengths

Run it on a rented GPU

On a cloud GPU with enough memory, the same command you'd use locally works:

ollama run llama4:scout

At Q4_K_M it needs about 68.3 GB, so pick a GPU with at least 72 GB.

Run it in the cloud

Too big for the NVIDIA RTX 4060? Run Llama 4 Scout 17B-16E on a rented GPU instead.

Rent a cloud GPU by the hour. No upfront cost, stop any time.

Rent a GPU on RunPodRent a GPU on Vast.ai

Memory needed by quantization

At 8,192 tokens of context. The black line marks the 8 GB the NVIDIA RTX 4060 can use.

QuantQualityMemoryFits?Speed
FP16Full quality220.2 GBToo big—
Q8_0Near-lossless118 GBToo big—
Q6_KExcellent91.6 GBToo big—
Q5_K_MVery good79.7 GBToo big—
Q4_K_MGood, most popular68.3 GBToo big—
Q3_K_MNoticeably degraded55.5 GBToo big—
Q2_KHeavily degraded43.1 GBToo big—

GPUs that run Llama 4 Scout 17B-16E comfortably

Models that run well on the NVIDIA RTX 4060

Frequently asked questions

Can NVIDIA RTX 4060 run Llama 4 Scout 17B-16E?

No. Even at Q4_K_M, Llama 4 Scout 17B-16E needs about 68.3 GB, but the NVIDIA RTX 4060 has 8 GB usable. You could offload some layers to system RAM, but it will be very slow.

How much VRAM does Llama 4 Scout 17B-16E need?

About 68.3 GB at Q4_K_M with 8,192 tokens of context, and about 220.2 GB at full FP16 precision.

How fast is Llama 4 Scout 17B-16E on the NVIDIA RTX 4060?

It does not fit in the NVIDIA RTX 4060's memory, so most layers would run from system RAM and generation would be very slow.