Run local AI on a Mac with oMLX: what is new in 0.7.1rc1?

The stable version is 0.7.0, while 0.7.1rc1 is a release candidate for testing new features. Earlier 0.7.1.dev1 changes and reproduction steps are retained below. Check the version in the admin UI or with omlx --version, and separately identify whether you installed the DMG app or from source. Update the app using a new bundle; for source installs, match that version’s dependencies and kernels.

oMLX runs MLX checkpoints², not GGUF³ files. It works on Apple Silicon Macs with macOS 15 or later. For a new installation, download the macOS DMG from the official Releases page, open it, move oMLX to the Applications folder, and launch it. Choose a model folder on the welcome screen, start the server, and download a compatible MLX checkpoint. Also check that the model format and quantization⁴ fit the Mac's unified memory⁵.

When the server is running, open the management UI at http://localhost:8000/admin and the chat UI at http://localhost:8000/admin/chat. Send a short question first to confirm that the model responds. If you are testing the development build, record your existing API⁶ connections and model folder so you can compare them with the previous setup.

After confirming a model response with a short question in the UI, check the basic API connection before moving on to performance comparisons. Replace YOUR_OMLX_API_KEY with the key configured in the management UI.

curl http://127.0.0.1:8000/health
curl http://127.0.0.1:8000/v1/models -H 'Authorization: Bearer YOUR_OMLX_API_KEY'

A healthy response from /health and a usable model ID from /v1/models confirm that the server and API authentication are ready. If the connection fails, first check that the server is running. If the model-list request returns 401, verify the key. If the list is empty, check the model folder and checkpoint download status.

Version 0.7.1rc1, published October 15, 2026, is a release candidate. The stable release is 0.7.0; choose rc1 to test the new features. Record your models and settings before updating so you can return to the stable release with the same model if needed.

You can run Qwen3.8 or Qwen3.6 through Splash while keeping model management and the API address in oMLX. Select the installed Splash launcher in Settings → Integrations → External engines, then enable Use external engine in the model’s Model Settings → Advanced. Place the matching DFlash2 draft in an oMLX model directory too.

Muse Spark Open is newly supported as well. The 370 GB oQ3e build targets a 512 GB Mac; select the model first, then budget unified memory and room for other apps. The updated downloader recommends models that can load on your Mac.

Prefill⁷ is the stage where a model reads a question and documents to prepare its first response. In 0.7.1.dev1, batched prefill combines chunks from multiple requests that begin input processing at similar times into one forward pass. It reduces repeated reads of MoE⁸ expert weights across requests while keeping each request's cache separate. No new setting is needed. This change is relevant on a server where requests from several users overlap.

The release table separates requests sent at once, requests spaced apart, and new requests joining ones that are already generating. When requests overlap, the next input can be prefilled alongside ongoing generation, so the effect is most relevant to first-token wait rather than single-request decode⁹ speed. Read the three patterns and hardware results in the table against your own request pattern.

On a personal Mac, first check whether requests actually overlap. Send the same prompt¹⁰ alone, then send overlapping requests and record first-token time and total completion time separately. Keep the model file, context, answer length, and number of concurrent requests the same. If even a single request has a slow first response, check input length, model loading, and memory state before focusing on batched-prefill results.

The group-size-32 Qwen3.8-Flash-Next checkpoint optimizes the decode path. A separate MTP-on comparison on M3 Ultra measured 98 to 134 tok/s (+37%). The Flash-Next row in the table is the MTP-off measurement.

INT8 activation prefill for Qwen3.8-27B oQ8e applies to the prefill stage that reads long inputs. On an M5 Max, the reported result for 4K input was +25%; the 16K condition is in the table. Enable it for models that show this option under Experimental Features, then record first-token wait for a long document separately from generation speed.

Lightning MTP¹¹ uses the model's next-token prediction head to propose candidates for verification by the main model, reducing sequential decode steps. The model families and file-editing conditions tested in the release are listed in the table. Enable it on a supported checkpoint, then compare completion time and output with standard generation on the same request.

The same release reports +31% decode for Qwen3.6-35B-A3B (M5 Max), +30% decode for GLM-5.3-Flash (M3 Ultra), and +48% on first run for DeepSeek V4.1 Flash (M3 Ultra). Match the stage and chip when reading each result, and keep those conditions in your test notes.

To try Qwen3.8-27B in another runtime¹², check the files and context length¹³ in TensorFold's Mac memory update. Use the same input and record memory use, time to the first reply and completion time.

If you update a source installation, oMLX 0.7.1.dev1 uses MLX 0.32.3. Kernels built with an earlier MLX version may be rejected and run through a slower fallback. Following the release notes, prepare the existing repository and virtual environment¹⁴, then run make kernels from the repository folder.

make kernels

When the build finishes without errors, restart oMLX. Check that Engine Versions in the management UI shows MLX 0.32.3, then send a short prompt you already use. A loaded model that returns a response confirms the basic path is working.

If make kernels fails, inspect the first error line and active Python virtual environment. If the command finishes but speed is lower, check the MLX version and kernel status in Engine Versions and look for kernel-loading errors in the server log. For an app bundle, use the official download for that installation type instead of this source procedure.

The release removes about 150 debug OMLX_* environment variables and no longer reads them. This includes OMLX_MOE_EXPERT_OFFLOAD, OMLX_QWEN35_ANE_PREFILL, and OMLX_RDMA_STAGE_LINKS. Adjust a feature through the model settings in the management UI when that model supports it.

For example, Experimental Features may show options such as INT8 activation prefill or ANE¹⁵ prefill only when the selected model supports them. Record the current setting, change one item, and send the same question. Compare input token¹⁶ count, first-token time, generation speed, and errors to see whether the setting affects your actual workload.

If the model list does not load, first check the model folder and download status. If the server opens but does not respond, look for the first error in the management UI status and server log. For a memory error, unload another model or reduce concurrent requests, then retry with a short input. If only speed changed, also check whether the input length, model file, and cache state were identical, not just the setting name.

Clef and OpenJev models return probabilities for choices after reading a state and its options, rather than generating free-form text each time. oMLX accepts TypeSafe System One requests at POST /v1/systemone. The question types are noul, choice, and score; oMLX can detect the checkpoint automatically, or you can set the model type to decision. This differs from a standard /v1/chat/completions request, so use it with an app or custom client that already supports this format.

EmbeddingGemma 2 now accepts image and audio inputs, and rerank requests add instruction and max_length options. The audio loaders improve ref_audio handling for Fish, TADA, and LongCat. Apps using these models should send images, audio, and reference voice inputs in the formats documented by their model cards and APIs.

The web dashboard has a new layout with detailed logs, links to settings sections, Restart and Unload all controls, and bundled benchmark presets. After updating, check model status in the new settings rail and Serving Stats area. Also locate model logs and restart controls in the management UI.

If concurrent requests are rare and work finishes reliably on 0.7.0, you can stay on the stable release. If overlapping requests make the wait for the first token a problem, measure TTFT¹⁷ on 0.7.1.dev1 with the same prompt and number of concurrent requests. If input processing or generation is slow for a specific Qwen checkpoint, check support conditions for group size, MTP, and INT8 prefill, then compare only on that model.

Record the installed version, model repository and quantization, Mac chip, input length, number of concurrent requests, MTP and cache settings, first-token time, completion time, and answer quality. Release figures compare against the code immediately before each change, not against 0.7.0 as a whole. Decide whether to update based on repeatable changes in your model and requests.

If a basic request fails, first get the server and model working before enabling more optimization options. Confirm a normal response to the same short request, then test features one at a time to narrow down which change caused the issue. Keep real work on the stable server, and use these records to decide whether to isolate development-build testing or migrate after confirming the new feature.