read first
Start using a local LLM: installation and your first model
Local LLM searches quickly lead to expensive GPUs and memory figures. Your first goal is smaller: watch your own computer answer something you wrote. After that, its limits become specific enough to make a useful decision.
Key takeaways
- 01
- Before installing, try a short question in our browser LLM.
- 02
- Consider LM Studio for a visual workflow or Ollama for repeatable commands.
- 03
- Allow memory for context and execution as well as weights. Start small.
Try one question before installing
Download the browser demo model and type a short question. The initial download needs internet; inference uses WebGPU in a supported browser. The demo uses Qwen 3.5 0.8B Q4, a starting point rather than a proxy for large-model quality.
Ask something you can judge, such as two simple meal ideas from ingredients you have. Starting with breaking news or specialist facts mixes knowledge quality with the basic question of whether inference works.
Check two memory figures
On Windows, check system memory and dedicated GPU memory in Task Manager. Do not add shared GPU memory as if it were dedicated VRAM. On a Mac, check the chip and memory in About This Mac; unified memory is shared with the rest of the system.
Download size is only a starting point. Longer conversations require more working memory, and other apps need their share. Begin with one small model and one short conversation so you can tell what changes actually solve a problem.
Choose one application first
LM Studio offers a visual search, download and chat flow. Ollama makes named-model commands and integrations convenient. You do not need both; use whichever lets you get a first answer with less friction.
Choose a model format supported by the application. GGUF, MLX and browser MLC files are not interchangeable just because the model name matches. The installation guides explain how to choose within each tool.
After it runs, repeat three useful questions
Keep three tests: edit a short sentence, summarize text you know, and answer a simple constrained question. Check length compliance, invented details and missed conditions as well as speed. Reusing these questions makes later model comparisons less subjective.
A slow first run may be model loading. A long wait before any words may be input processing; slowly appearing words point to decode speed. These are different problems, even if all feel slow.
When does an upgrade become useful?
If a small model keeps missing constraints, try a more capable one. If the answer is good but slow, throughput matters. If failures appear only with longer documents, examine context and memory. Separating these cases makes an upgrade decision concrete.
In the comparison tool, keep model and input length fixed while changing hardware. Read the hardware guide for budget and expansion choices, or a serving recipe to improve the machine you own. A successful first question is already a useful first session.