Local LLM not using your GPU?
If your graphics card sits idle while the CPU does all the work, one of these is usually the reason.
Sounds like this?
- Task Manager shows the CPU at 100% and the GPU near 0%
ollama pssays "100% CPU"- Speeds of 1 to 5 words a second on a fast gaming GPU
1. Update your graphics driver
NVIDIA: install the latest Game Ready or Studio driver from nvidia.com or the NVIDIA app, then restart. AMD: install the latest Adrenalin driver from amd.com. Old drivers are the most common cause.
2. Laptops with two GPUs
Many laptops have a weak built-in GPU and a powerful one. In Windows, open Settings > System > Display > Graphics, add LM Studio or Ollama, and set it to High performance.
3. AMD cards
LM Studio uses Vulkan, which works with most Radeon cards, so it is the easier choice on AMD. Ollama supports newer Radeon cards (RX 6000 series and later) through ROCm. Check Ollama's supported GPU list.
4. Check the app's GPU setting
In LM Studio, open the model's load settings and set GPU Offload to the maximum. In the app settings, under Runtime, choose the CUDA (NVIDIA) or Vulkan (AMD) engine. In Ollama, if the model is bigger than your graphics memory, some or all of it runs on the CPU: try a smaller model.
5. On Linux
Install the proprietary NVIDIA driver and check that nvidia-smi lists your card. If Ollama was installed before the driver, reinstall Ollama so it picks up GPU support.
6. Intel Macs
Local AI tools only use the GPU on Apple Silicon Macs (M1 and newer). On an Intel Mac models run on the CPU, so stick to small ones such as Llama 3.2 3B, or rent a GPU.
Check what fits first
Most problems come from a model that's too big. See what fits your GPU or Mac, and how fast it will run.
Run it in the cloud
Still too big or too slow? Run it on a rented GPU.
A rented GPU with 24 to 80 GB runs models your computer can't, from well under a dollar an hour.