Rent a GPU

Free LLM VRAM calculator

Can your GPU or Mac run that AI model locally?

Pick a model and your hardware. See if it fits, which quantization to download, and how fast it will run.

New to local AI? Let us pick a model for you

“From a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer.”

Sam Altman, OpenAI CEO, on Cheeky Pint, October 2026. We think your own hardware is a fine place for it.
  • 69open models
  • 51GPUs and Macs
  • 3519checked pairs
gpufit / estimateLive
More options

KV cache: set OLLAMA_KV_CACHE_TYPE in Ollama (needs flash attention) or the K/V cache option in LM Studio. Click a quantization below to see its numbers.

YesRuns fully on the GPU
Memory at Q5_K_M7.4 GB of 8 GB
Weights 5.7 GBKV cache 1.1 GBRuntime 0.6 GB
QuantQ5_K_MVery good
Speed~29tokens per second
Max context11Ktokens that fit at Q5_K_M

Run it with one command in Ollama:

ollama run llama3.1:8b
See the full breakdown

Run it in the cloud

Rent a cloud GPU when your card is too small

No GPU, or not enough VRAM? Rent one by the hour and run any model today. No upfront cost, stop any time.

RunPod
  • On-demand GPUs billed by the second
  • Ready-made Ollama and vLLM templates
  • RTX 4090 up to H100 80 GB
Rent a GPU on RunPodVast.ai
  • Marketplace of rentable GPUs
  • Often the lowest hourly prices
  • Consumer RTX cards to data-centre GPUs
Rent a GPU on Vast.ai

What each memory size runs

The largest model we track that runs well at Q4_K_M or better, 8K context.

MemoryLargest modelExample GPU
8 GBQwen3 8BRTX 5060
12 GBQwen3 14BRTX 3080 Ti
16 GBgpt-oss 20BRTX 5080
24 GBQwen3.6 35B-A3BRTX 4090
32 GBMixtral 8x7BRTX 5090
48 GBQwen2.5 72BRent: RTX A6000, L40S
80 GBgpt-oss 120BRent: A100, H100

Find your GPU or your model

Every page lists what runs, at which quantization, and how fast.

NVIDIA 26

AMD 9

Intel 2

Apple 14

Llama 6

Qwen 23

DeepSeek 8

Mistral 8

Gemma 9

Phi 4

gpt-oss 2

Granite 1

GLM 5

Kimi 1

MiniMax 1

Nemotron 1

How GPU Fit estimates memory and speed

Token generation is limited by memory size and memory bandwidth, so both can be estimated from the spec sheet.

  • WeightsParameters × bits per weight of the quantization.
  • KV cache2 × layers × KV heads × head size × context × 2 bytes, with sliding-window layers capped at their window.
  • Speed60% of peak bandwidth ÷ bytes of active weights per token.

Read the full methodology

InputModel + GPU + context
EstimatorBest quant that fits
Weightsparams × bits
KV cachelayers × heads × context
Speedbandwidth ÷ weights

Ask from your AI agent

The same estimates over MCP: check_fit, models_for_gpu, gpus_for_model.

{
  "mcpServers": {
    "gpufit": { "type": "http", "url": "https://gpufit.dev/mcp" }
  }
}

Questions about running LLMs locally

Short answers. The calculator above does the exact numbers for your hardware.

How much VRAM do I need to run an LLM locally?

At the popular Q4_K_M quantization, plan on roughly 0.6 GB per billion parameters plus 1 to 2 GB for context and overhead: about 6 GB for an 8B model, 10 GB for 14B, 21 GB for 32B and 44 GB for 70B at 8K context. Longer context needs more.

Which quantization should I use?

Use the highest quality quantization that fits fully in GPU memory. Q4_K_M is the usual sweet spot; Q5_K_M, Q6_K and Q8_0 are better if you have room; Q3 and Q2 noticeably hurt quality.

Can a Mac run local LLMs?

Yes. Apple Silicon Macs share memory between CPU and GPU, and macOS lets the GPU use about two thirds to three quarters of it by default, so a 64 GB or 128 GB Mac can run models that would need several consumer GPUs.

How are tokens per second estimated?

Generating text is limited by memory bandwidth: each token reads all active weights once. We estimate speed as 60% of the GPU's peak bandwidth divided by the size of the active weights.

What if my GPU is too small?

Use a smaller model or a lower quantization, offload some layers to system RAM (slow), or rent a cloud GPU by the hour.