Rent a GPU

Free LLM VRAM calculator

Can your GPU or Mac run that AI model locally?

Pick a model and your hardware. See if it fits, which quantization to download, and how fast it will run.

  • 29open models
  • 24GPUs and Macs
  • 696checked pairs
gpufit / estimateLive
YesRuns fully on the GPU
Memory at Q5_K_M7.4 GB of 8 GB
Weights 5.7 GBKV cache 1.1 GBRuntime 0.6 GB
QuantQ5_K_MVery good
Memory7.4 GBof 8 GB usable
Speed~29tokens per second
  1. FP16
  2. Q8_0
  3. Q6_K
  4. Q5_K_M
  5. Q4_K_M
  6. Q3_K_M
  7. Q2_K
See the full breakdown

Run it in the cloud

Rent a cloud GPU when your card is too small

No GPU, or not enough VRAM? Rent one by the hour and run any model today. No upfront cost, stop any time.

RunPod
  • On-demand GPUs billed by the second
  • Ready-made Ollama and vLLM templates
  • RTX 4090 up to H100 80 GB
Rent a GPU on RunPodVast.ai
  • Marketplace of rentable GPUs
  • Often the lowest hourly prices
  • Consumer RTX cards to data-centre GPUs
Rent a GPU on Vast.ai

What each memory size runs

The largest model we track that runs well at Q4_K_M or better, 8K context.

MemoryLargest modelExample GPU
8 GBQwen3 8BRTX 3070
12 GBQwen3 14BRTX 5070
16 GBgpt-oss 20BRTX 5080
24 GBQwen2.5 32BRTX 4090
32 GBMixtral 8x7BRTX 5090
48 GBQwen2.5 72BRent: RTX A6000, L40S
80 GBgpt-oss 120BRent: A100, H100

Find your GPU or your model

Every page lists what runs, at which quantization, and how fast.

NVIDIA 15

AMD 2

Apple 7

Llama 5

Qwen 10

DeepSeek 1

Mistral 4

Gemma 5

Phi 2

gpt-oss 2

How GPU Fit estimates memory and speed

Token generation is limited by memory size and memory bandwidth, so both can be estimated from the spec sheet.

  • WeightsParameters × bits per weight of the quantization.
  • KV cache2 × layers × KV heads × head size × context × 2 bytes.
  • Speed60% of peak bandwidth ÷ bytes of active weights per token.

Read the full methodology

InputModel + GPU + context
EstimatorBest quant that fits
Weightsparams × bits
KV cachelayers × heads × context
Speedbandwidth ÷ weights

Ask from your AI agent

The same estimates over MCP: check_fit, models_for_gpu, gpus_for_model.

{
  "mcpServers": {
    "gpufit": { "type": "http", "url": "https://gpufit.dev/mcp" }
  }
}

Questions about running LLMs locally

Short answers. The calculator above does the exact numbers for your hardware.

How much VRAM do I need to run an LLM locally?

At the popular Q4_K_M quantization, plan on roughly 0.6 GB per billion parameters plus 1 to 2 GB for context and overhead: about 6 GB for an 8B model, 10 GB for 14B, 21 GB for 32B and 44 GB for 70B at 8K context. Longer context needs more.

Which quantization should I use?

Use the highest quality quantization that fits fully in GPU memory. Q4_K_M is the usual sweet spot; Q5_K_M, Q6_K and Q8_0 are better if you have room; Q3 and Q2 noticeably hurt quality.

Can a Mac run local LLMs?

Yes. Apple Silicon Macs share memory between CPU and GPU, and macOS lets the GPU use about two thirds to three quarters of it by default, so a 64 GB or 128 GB Mac can run models that would need several consumer GPUs.

How are tokens per second estimated?

Generating text is limited by memory bandwidth: each token reads all active weights once. We estimate speed as 60% of the GPU's peak bandwidth divided by the size of the active weights.

What if my GPU is too small?

Use a smaller model or a lower quantization, offload some layers to system RAM (slow), or rent a cloud GPU by the hour.