Response speed and acceleration
Last updated
Is the Neural Engine actually working? Inspect Core ML execution
Inspect operation placement alongside completion time.
Open a Core ML1 model in Xcode to inspect which operations run on the CPU2, GPU3, or Neural Engine4 and how long loading and prediction take. Compare All, CPU and GPU, and CPU and Neural Engine for the same model to choose a configuration on your Mac.
You need Xcode and an .mlmodel or .mlpackage to inspect. A report can be generated by opening the file without adding it to an app project. For GGUF5 or MLX6 files, compare through that runtime7's logs; use this procedure when you have a Core ML conversion. Apple performance inspection
Open the model and create an All report
Open the model in Xcode and switch to Performance. Click the Add button at the lower left, select My Mac, choose All for Compute Units, and run the test. The completed report shows Prediction, Load, Compilation, and operation placement.
Close unnecessary apps so other work does not interfere during the test. Record the device name, macOS and Xcode versions, and model filename with the report. Keep the model file unchanged and vary only Compute Units to compare the effect of different execution devices.
Compare allowed compute-unit configurations
Create two more reports for the same model: CPU and GPU, and CPU and Neural Engine. Core ML places operations among the permitted devices, so read timing and placement together. Core ML execution configuration
| Setting | Allowed devices | Purpose |
|---|---|---|
| All | CPU, GPU, ANE | Inspect default placement |
| CPU and GPU | CPU·GPU | GPU-path baseline |
| CPU and Neural Engine | CPU·ANE | Compare ANE participation |
Find operations marked as executing in the report's ANE column. In the All configuration, also inspect whether expensive operations are placed on the CPU, GPU, or ANE, then compare Prediction time with the other two settings. Choose the faster setting after checking that its output is correct.
Separate startup from repeated execution
Compilation prepares a model for execution, Load brings the prepared model into use, and Prediction computes the input. If the first run takes a long time, record downloads separately and inspect these three report metrics. Your wait also depends on whether the app reloads the model each time or keeps it loaded.
The report offers statistics such as Median and First. Record the repeated-run median for steady use, and first execution plus uncached loading if the app is restarted often. Prediction timing from an already-loaded model omits the wait a user experiences when opening the app.
Check the same input in the actual app
After finding a fast configuration, process the same recording or image in the app. Record completion time including preprocessing, file reading, and displaying the result. For transcription, also inspect omissions and misrecognized words before deciding to keep an optimization.
If a report cannot be generated, check that the model download is complete and that your macOS and Xcode versions support it. A CPU only report can establish whether the model itself runs. If only one device fails, record that report's error and the model version to investigate.
Terminology notes
Core ML — Apple’s framework for running machine-learning models on its devices.
Back to the textCPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textNeural Engine — Dedicated hardware in Apple Silicon for neural-network computation.
Back to the textGGUF — A file format for model data, widely used by llama.cpp-based tools.
Back to the textMLX — Apple’s machine-learning framework, which runs models using memory shared by the CPU and GPU on Apple Silicon.
Back to the textRuntime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the text
Frequently asked questions
- What should you check first in this article?
- Open a Core ML model in Xcode to inspect which operations run on the CPU, GPU, or Neural Engine and how long loading and prediction take. Compare All, CPU and GPU, and CPU and Neural Engine for the same model to choose a configuration on your Mac.
- When is it better not to choose this?
- Close unnecessary apps so other work does not interfere during the test. Record the device name, macOS and Xcode versions, and model filename with the report. Keep the model file unchanged and vary only Compute Units to compare the effect of different execution devices.