Rent a GPU

Phi-4 Mini 3.8B hardware requirements

3.84B parameters. Needs about 4 GB at Q4_K_M with 8K context.

Parameters3.84BDense
Q4_K_M · 8K4 GBthe usual download
FP16 · 8K9.4 GBfull precision
Runs well on51/51GPUs and Macs we track

Memory needed at each quantization

Weights, KV cache and runtime overhead, at 8K and 32K tokens of context.

QuantQuality8K context32K context
FP16Full quality9.4 GB12.6 GB
Q8_0Near-lossless5.8 GB9 GB
Q6_KExcellent4.8 GB8 GB
Q5_K_MVery good4.4 GB7.6 GB
Q4_K_MGood, most popular4 GB7.2 GB
Q3_K_MNoticeably degraded3.6 GB6.8 GB
Q2_KHeavily degraded3.1 GB6.3 GB

Run it with Ollama

Downloads Q4_K_M by default.

ollama run phi4-mini:3.8b

Model card

microsoft/Phi-4-mini-instruct on Hugging Face: licence, prompt format and GGUF builds.

Run it in the cloud

Run Phi-4 Mini 3.8B on a rented GPU today.

It needs about 4 GB at Q4_K_M: a rented RTX 3090 / RTX 4090 runs it for about $0.18 to $0.89 an hour.

RunPodRent a GPU on RunPodVast.aiRent a GPU on Vast.ai

Every GPU and Mac checked

Sorted by memory. Bars show how much of each card Phi-4 Mini 3.8B fills at the best quant that fits.

51 Runs well

NameMemory usedBest quantSpeed

Other Phi models