gpt-oss 20B hardware requirements
20.9B parameters (3.6B active, mixture of experts). Needs about 13.7 GB at Q4_K_M with 8K context. Model card.
Memory needed at each quantization
| Quant | Quality | 8K context | 32K context |
|---|---|---|---|
| FP16 | Full quality | 42.8 GB | 44 GB |
| Q8_0 | Near-lossless | 23.2 GB | 24.4 GB |
| Q6_K | Excellent | 18.1 GB | 19.3 GB |
| Q5_K_M | Very good | 15.9 GB | 17.1 GB |
| Q4_K_M | Good, most popular | 13.7 GB | 14.9 GB |
| Q3_K_M | Noticeably degraded | 11.2 GB | 12.4 GB |
| Q2_K | Heavily degraded | 8.8 GB | 10 GB |
Run it with Ollama: ollama run gpt-oss:20b (downloads Q4_K_M by default).
Run it in the cloud
Don't want to buy hardware? Run gpt-oss 20B on a rented GPU today.
Rent a cloud GPU by the hour. No upfront cost, cancel any time.
Every GPU and Mac checked
| GPU | Runs? | Best fit |
|---|---|---|
| NVIDIA RTX 3060 12GB | Yes, with heavy quantization | Q3_K_M · ~11.2 GB · ~123 tok/s |
| NVIDIA RTX 3070 | No | Needs ~13.7 GB at Q4_K_M; 8 GB available |
| NVIDIA RTX 3080 10GB | Yes, with heavy quantization | Q2_K · ~8.8 GB · ~338 tok/s |
| NVIDIA RTX 3090 | Yes | Q8_0 · ~23.2 GB · ~147 tok/s |
| NVIDIA RTX 4060 | No | Needs ~13.7 GB at Q4_K_M; 8 GB available |
| NVIDIA RTX 4060 Ti 16GB | Yes | Q5_K_M · ~15.9 GB · ~67 tok/s |
| NVIDIA RTX 5060 Ti 16GB | Yes | Q5_K_M · ~15.9 GB · ~105 tok/s |
| NVIDIA RTX 4070 | Yes, with heavy quantization | Q3_K_M · ~11.2 GB · ~172 tok/s |
| NVIDIA RTX 4070 Ti Super | Yes | Q5_K_M · ~15.9 GB · ~157 tok/s |
| NVIDIA RTX 4080 | Yes | Q5_K_M · ~15.9 GB · ~168 tok/s |
| NVIDIA RTX 4090 | Yes | Q8_0 · ~23.2 GB · ~158 tok/s |
| NVIDIA RTX 5070 | Yes, with heavy quantization | Q3_K_M · ~11.2 GB · ~229 tok/s |
| NVIDIA RTX 5070 Ti | Yes | Q5_K_M · ~15.9 GB · ~210 tok/s |
| NVIDIA RTX 5080 | Yes | Q5_K_M · ~15.9 GB · ~225 tok/s |
| NVIDIA RTX 5090 | Yes | Q8_0 · ~23.2 GB · ~281 tok/s |
| AMD Radeon RX 7800 XT | Yes | Q5_K_M · ~15.9 GB · ~146 tok/s |
| AMD Radeon RX 7900 XTX | Yes | Q8_0 · ~23.2 GB · ~151 tok/s |
| Apple M4 (16GB) | Yes, with heavy quantization | Q2_K · ~8.8 GB · ~53 tok/s |
| Apple M1 (16GB) | Yes, with heavy quantization | Q2_K · ~8.8 GB · ~30 tok/s |
| Apple M2 Pro (32GB) | Yes | Q6_K · ~18.1 GB · ~41 tok/s |
| Apple M3 Max (64GB) | Yes | FP16 · ~42.8 GB · ~33 tok/s |
| Apple M4 Pro (48GB) | Yes | Q8_0 · ~23.2 GB · ~43 tok/s |
| Apple M4 Max (128GB) | Yes | FP16 · ~42.8 GB · ~45 tok/s |
| Apple M2 Ultra (192GB) | Yes | FP16 · ~42.8 GB · ~67 tok/s |