How the estimates work

Memory = model weights (parameters × bits per weight for the quantization) + KV cache (2 × layers × KV heads × head size × context × 2 bytes) + ~0.6 GB runtime overhead.

Usable memory: the full VRAM on discrete GPUs. On Apple Silicon, macOS lets the GPU use about 67% of unified memory (75% above 36 GB) by default.

Speed: token generation is memory-bandwidth bound, so we estimate tokens/second as 60% of peak bandwidth divided by the bytes of active weights read per token.

These are estimates. Real results depend on runtime, drivers, batch size and settings.

For AI agents (MCP)

AI assistants and agents can use these estimates directly through our Model Context Protocol server at https://gpufit.dev/mcp (Streamable HTTP, no sign-in). Tools: check_fit, models_for_gpu, gpus_for_model and list_catalog.

Affiliate disclosure

Some links (GPU clouds, retailers) are affiliate links. If you buy or sign up through them we may earn a commission at no extra cost to you. It does not change the estimates.