memory and model
M5 Ultra SSD: faster loading or faster local AI?
When shopping for an M5 Ultra Mac Studio, the much faster SSD1 sounds like another reason to upgrade. Then comes a tempting question: if storage is this fast, could you buy less memory and spend more on the SSD? The answer changes when you separate opening a model for the first time from continuing a conversation with one already loaded.
Where does a faster SSD actually save time?
Selecting a model in a chat app does not produce an answer immediately. First the runtime2 makes the stored model ready to use, then processes your question and documents, and finally generates tokens3. All three feel like waiting, but they are different jobs inside the computer.
SSD improvements matter most directly in the first stage. Someone switching between coding, image and chat models may repeat it many times a day. Someone who keeps a single model loaded may appreciate the shorter startup but notice little difference in the conversations that follow.
| What you see | Main task | Role of the SSD |
|---|---|---|
| Opening the model | Read weights and prepare execution | Directly affects actual disk reads |
| Waiting for the first token | Prefill and cache handling | Compute and context matter when weights are resident |
| The answer keeps appearing | Decoding successive tokens | Resident inference mainly depends on memory and compute |
Three kinds of waiting in a local LLM
Opening the model
- Main task
- Read weights and prepare execution
- Role of the SSD
- Directly affects actual disk reads
Waiting for the first token
- Main task
- Prefill and cache handling
- Role of the SSD
- Compute and context matter when weights are resident
The answer keeps appearing
- Main task
- Decoding successive tokens
- Role of the SSD
- Resident inference mainly depends on memory and compute
What a roughly 14GB/s result tells you
Tom's Guide measured Blackmagic reads of 13,902MB/s and writes of 14,283MB/s on its M5 Ultra with a 4TB SSD. Those are disk results for that configuration, not a guarantee across capacities or a multiplier for LLM4 generation speed.
Suppose you need to read 70GB. At a steady 7GB/s, the transfer takes ten seconds; at 14GB/s, five. This is an illustrative transfer-only calculation, not a loading benchmark. File handling, memory allocation and runtime preparation add work, so the app's total wait need not halve.
A faster second load is not necessarily the SSD showing its full speed. The operating system may have kept recently read file data in memory. That file cache is separate from a prompt cache, which reuses prior prompt computation. Treating both as a single cold-versus-warm condition confuses disk and inference5 performance.
If the model is loaded but answers are still slow
If the model is resident in unified memory6 without swapping or offload7, generation mostly moves data between memory and compute hardware, not the SSD. Retrieving a book from a fast archive does not determine how quickly you read it once it is on your desk. Disk bandwidth therefore cannot be converted directly into tokens per second.
If nothing appears for a while and then the answer streams quickly, look at input length first: prefill8 may be processing a long document or conversation. If the first token arrives promptly but the rest crawls, examine model size, quantization9, runtime and decoding10 acceleration. The same SSD upgrade is not a useful prescription for both.
This site's speed experience separates prefill from decoding. Its wait time is not a measurement of initially loading model files from the SSD. Use it to compare responses at your input length once the model is ready to run.
The important exception: keeping part of a model on SSD
Not every model fits fully in memory. With a supported model and runtime, PLE11 offload can leave large lookup tables on SSD and read only the required entries. Think of retrieving selected index cards rather than moving the entire collection each time. Storage performance matters during generation in this case because disk access remains part of the work.
MacStories ran Qwen3.8-Flash-Next oQ8e with PLE tables on SSD on an 80-GPU12-core M5 Ultra with 256GB. With oMLX 0.7.0.dev2 and MTP13 depth 3, prose generation had a median of 86.8 tok/s; sampled process memory during answers was about 187GB, not weights alone. This demonstrates a working configuration, not a controlled SSD-only comparison.
MTP and PLE offload play different roles. MTP proposes future tokens to reduce generation iterations; PLE offload changes where selected data is stored. They can coexist when the runtime supports that combination. This does not make PLE a universal model option or the same feature as ordinary n-gram14 speculative decoding15.
Reading a large file sequentially differs from fetching small scattered entries. Reuse through the operating system's file cache also matters. A twofold disk benchmark improvement is therefore not a basis for predicting twice the offloaded decoding speed.
Can 256GB plus a fast SSD replace 512GB of memory?
It can be an option for a particular model, but not a general replacement rule. Some models lack selective offload support; long-context KV caches and other loaded models need memory too. Comparing a model's file size with the nominal 256GB capacity does not settle whether the whole workload fits.
Operating-system swap is another case. Unlike selective PLE offload, memory pressure can repeatedly evict data that execution soon needs again. That can cause stalls and uneven response times. Faster storage may reduce the penalty, but does not reproduce the behavior of having enough memory.
Conversely, if your model and context already fit comfortably, moving to 512GB does not automatically double generation speed. Its value is room for larger models, longer contexts or multiple models. List what you will keep running every day before buying for the largest model you might someday try.
Spend on memory first—unless your bottleneck is elsewhere
If the daily model barely fits, prioritize memory. Faster loading solves little if the workload cannot run reliably. Allow room for the model, KV cache16 and other applications first, then choose storage for frequently used weights and working files.
If memory is already sufficient and you alternate between several large models, faster storage becomes more valuable: you repeatedly pay the loading cost. Frequent image-checkpoint17 changes are a similar case. Still, do not assume a larger SSD is automatically faster; look for measurements of the capacity you intend to buy.
An external SSD can make sense for archiving models you only occasionally try. Keep daily weights internally and less-used models elsewhere. External performance depends on the drive, enclosure, connection, cable and thermals together; a Thunderbolt 5 label does not establish parity with the internal SSD.
Already own a Mac? Check this before upgrading
Use one regular model and question. Observe model loading, time until the answer starts, and generation speed separately. Check memory pressure and swap in Activity Monitor. A single accumulated swap value is not enough: watch whether it rises during the task and coincides with response stalls.
For a side-by-side test, match model files, quantization, input length, runtime version and MTP settings. Record file-cache and prompt-cache conditions too. Once you distinguish painful model switching from slow long-document processing or insufficient memory, the right upgrade becomes much clearer.
The SSD does not have to be your main reason to choose M5 Ultra
It is natural to hope that better hardware will remove the friction you feel today. What matters is where that friction comes from. Long conversations with one model prioritize memory fit and response speed; frequent switching adds loading time to the decision. The right M5 Ultra configuration need not be the same for both people.
Start with the model you intend to use and experience its prefill and decoding. If that speed is sufficient and memory has room to spare, choose SSD capacity around storage needs and switching convenience. Spending on the part that actually keeps you waiting is more useful than buying the largest number.
Terminology notes
SSD — A data storage device that uses flash memory. It retains data without power and serves a different role from system memory.
Back to the textRuntime — Software that loads model files and runs their computations. Supported formats, hardware, and optimizations vary by runtime.
Back to the textToken — A unit into which a model divides input or output for processing. One token does not equal one character or a fixed duration.
Back to the textLarge language model — A language model trained on large text datasets to process and generate text. Capabilities and supported inputs vary by model.
Back to the textInference — The process of using a trained model to compute an output for an input. Here, local inference means running the model on the user’s device.
Back to the textUnified memory — An architecture where the CPU and GPU share one physical memory pool. It does not increase total memory capacity; available capacity depends on the system.
Back to the textOffloading — Moving some model data from GPU memory to system RAM or storage when capacity is limited. This adds data transfer.
Back to the textPrefill — The stage where an LLM reads the input prompt and computes representations for its tokens. Longer prompts contain more tokens to process.
Back to the textQuantization — Representing model values with fewer bits. Memory use, accuracy, or execution speed may change; the effects depend on the format and implementation.
Back to the textDecode — For an LLM, this is the stage that generates output tokens after input processing. For a VAE or audio codec, decoding can mean reconstructing the original form from a compressed representation or encoded data.
Back to the textPLE embedding table — Short for Per-Layer Embedding, a structure that adds embedding information at particular model layers. The Qwen Flash-Next family uses n-gram tables; supported runtimes can offload these tables to RAM or SSD.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textMTP — A training method or model component for predicting multiple future tokens. Whether it improves generation speed depends on implementation and runtime conditions.
Back to the textN-gram — A sequence of n consecutive tokens or items. N-grams may be used for context lookup or speculative generation, and may also form the entries of embedding tables in some models.
Back to the textSpeculative decoding — A generation method that proposes output candidates for the main model to verify. Candidates may come from a separate draft model, an MTP head, or lookup of repeated context; speed effects depend on implementation and conditions.
Back to the textKV cache — Memory that stores attention keys and values from earlier tokens for reuse during later token generation. Its size depends on context length and batch size.
Back to the textCheckpoint — A file containing saved model weights and related state. Versions or tasks in one model family may use different checkpoints.
Back to the text