Which local LLM app or serving engine should you start with?

Do you want to chat, or call the model from another program?

If you mainly want to ask questions and read answers, choose a tool that makes finding and loading a model straightforward. LM Studio makes it convenient to inspect files and runtime status on screen. Ollama offers a simple flow for managing named models and using a local API. Neither requires you to add a separate web chat interface at the outset.

An editor or coding agent needs the correct server address and model name. A working chat does not by itself prove that tool calls work. Test a short question, streaming and tool calls in that order in the app you intend to use. API compatibility is not a reason to skip these checks.

• LM Studio: choose a model on screen and connect a local server
• Ollama: get started, then check where the model is running

On a Mac, check the file before the engine

Two downloads carrying the same Qwen name may be a GGUF file and an MLX conversion. Even if both say 4-bit, their quantization and retained weights are not necessarily identical. If you change the model file while trying to compare engines, it becomes harder to tell what caused the difference.

For MLX on Apple Silicon, oMLX and MTPLX are worth examining. The oMLX guide covers model management and caching repeated input; the MTPLX guide covers MTP on supported models and choosing a draft depth on your Mac. Read whether your existing file can be used first. If your current workflow works well, there is no need to install every engine.

• oMLX: manage models and reduce the wait on long input
• MTPLX: check whether MTP helps on your Mac

Consider llama.cpp when you want direct control

The llama.cpp server is an option when you want to control how much of the model goes on the GPU, how much context is allocated and whether part of the work runs on the CPU. You will also take over file management and launch options that an app might otherwise handle. Before copying a long command from another guide, finish a short request with one of your own files.

The benefit is not a guarantee of the highest speed on every device. It is being able to record exactly what you ran. Save a working command and change one setting at a time; then you can trace why memory ran out or the first response became slower. Those records also tell you what to match when comparing hardware later.

• llama.cpp: start a server with one GGUF file

Queued requests change what you need from a server

When chatting alone, you care about when the first text appears and when the answer finishes. If document processing and a coding agent submit requests at the same time, the queue matters too. The case for exploring vLLM or SGLang becomes clearer when you need to keep a model serving and manage those requests.

With multiple GPUs, first distinguish splitting one model across devices from loading separate copies and dividing the requests. Do not assume ordinary PC installation commands will transfer unchanged to Spark's ARM64 environment. The short examples below check connectivity; larger-model quantization and distributed settings belong in the hardware-specific recipes.

• vLLM: separate single-response speed from server throughput
• SGLang: check repeated requests and model-specific execution paths

Give the old and new setup the same job

Keep a question and a document from your usual work. Record the model file, input length and output limit, and distinguish reading a document for the first time from reading it again. Compare the wait before output, the pace of generation and the parts of the answer you still had to fix. This makes the decision easier than relying on one number.

If the new engine finishes sooner but produces a shorter answer or omits a tool call, it may not have completed the same job. Conversely, similar speed can still be worthwhile if easier model switching makes you use it more often. Choosing an engine is less about memorizing a winner and more about finding the inconvenience you want to remove.