Run your first local AI model
Answer two questions and we'll pick one model that suits your computer, then walk you through setting it up. No jargon, about ten minutes, and it all runs privately on your machine.
- What for
- Your computer
- Your model
What do you want to use it for?
Pick the closest. You can change it later.
What computer are you on?
Reads your graphics card's name from the browser. Nothing is sent anywhere.
Not sure? On Windows open Task Manager > Performance > GPU: the card's name is at the top right and its memory is "Dedicated GPU memory".
Not sure? Click the Apple menu > About This Mac. It shows your Chip and Memory.
Not sure? On Windows open Settings > System > About and look at "Installed RAM".
The words you'll see, in plain English
You don't need to know these to get started, but they help when you read about models.
- Model
- The AI itself: one big file you download, a few GB to over 100 GB. Llama, Qwen, Gemma and gpt-oss are families of models from different companies.
- Parameters (the "B" number)
- "8B" means 8 billion parameters. More parameters usually means smarter, but also bigger and slower.
- Graphics memory (VRAM)
- The memory on your graphics card. The whole model needs to fit in it to run fast. On Apple Silicon Macs, the CPU and GPU share one pool called unified memory.
- Quantization (Q4_K_M and friends)
- A way to shrink a model by storing its numbers with less precision. Q4_K_M is about a quarter of the original size with little loss in quality, which is why it's the default download.
- Tokens per second
- How fast the model writes. A token is about three quarters of a word. Around 10 tokens a second is comfortable reading speed.
- Context length
- How much text the model can keep in mind at once, counting your messages, its replies and any documents you add. Longer context uses more memory.
LM Studio or Ollama?
Both are free, run the same models, and keep everything on your computer.
LM Studio
Easiest for beginners. A normal app with a chat window: you search for a model, click download, and start typing. No commands.
Ollama
One command downloads and runs a model. Good if you are comfortable with a terminal, or want other apps and coding tools to use your local model.
Common first questions
Is running AI locally free?
Yes. LM Studio, Ollama and the models we recommend are free to download and use. You only pay for the electricity your computer uses.
Is it private?
Yes. Once the model is downloaded, your chats stay on your computer and work without an internet connection.
Will it be as good as ChatGPT?
The biggest open models come close on many everyday tasks; small ones that fit on a laptop are noticeably weaker. The picker chooses the biggest model that runs at a comfortable speed on your machine.
Can it damage my computer?
No. It works your graphics card about as hard as a game does. A large model can make other apps sluggish while it runs; closing the chat app frees the memory again.