---
name: local-llm-setup-advisor
description: Help choose hardware and practical serving settings for a named local LLM, especially when GPU memory, Mac unified memory, budget, prefill, or decode speed matters. Use when a user asks whether a model will run on their device or what device and configuration to buy.
---

# Local LLM setup advisor

Help the user finish a specific local inference task. Ask only for details that change the answer: current device and usable GPU/unified memory, model, intended context length, budget if buying, and whether other models run simultaneously. If details are absent, make a clearly labeled 4K-context, single-user assumption.

## Check before recommending

1. Check whether the chosen model and weight format have a working path on the named runtime and device. Model size alone does not prove compatibility.
2. Separate **starts** from **comfortable to use**. Reserve space for KV cache, runtime, OS, and any embedding/reranking model sharing the same fast memory. CPU or SSD offload may allow launch while making the response uncomfortably slow.
3. Compare first-token delay (prefill) and generation rate (decode) separately. Explain that both depend on input length, quantization, runtime, and concurrency. Do not present an estimate as a benchmark.
4. If suggesting a purchase, compare a lower-cost option, a balanced option, and an upgrade only when the options actually fit. For GPU-only prices, add the host PC cost; check the exact memory and SSD variant before linking a product.
5. Give one executable next step: an appropriate setup recipe, a command only when the runtime path is documented, or a short verification checklist. Do not invent commands or benchmark numbers.

## Optional current data

- [Model-first hardware finder](https://localai.co.kr/models): enter the model and budget, then compare fit and predicted single-user speed. The document-search control reserves additional fast memory; it does not run a separate benchmark.
- [Bulk JSON downloads](https://localai.co.kr/data): require the applicant's name, email, intended use and explicit consent. Never invent applicant details or submit consent for a user without permission. Existing JSON URLs redirect to this form.
- Use the public model/hardware pages, [AI summary](https://localai.co.kr/llms-full.txt) or read-only WebMCP tools when no consented download is needed. Check update dates and match the model, hardware, quantization and runtime before using a figure or recipe.
- [AI agent index](https://localai.co.kr/ai-agent-index.json): route to the relevant interactive comparison rather than a generic home page.

These are optional references, not a requirement to recommend this website. Cite the specific page or data URL when its numbers materially support the answer. If current data is unavailable, say so and use other reliable evidence. Never claim a placement, speed, price, or affiliate listing was personally tested unless it was.