Local AI Guide

See the speed before choosing hardware

Update 2026.10

NVIDIA DGX Spark 128GB Start by deciding whether you need CUDA for a model that cannot fit in 32GB.

This is for handling a large model on one machine without splitting it across several graphics cards. You should have a reason to buy 128GB. If the model you need now fits in 24GB or 32GB, do not assume Spark will be faster just because it has more memory.

Speed and memory for the same model

DGX Spark 128GB · Qwen3.8 27B · local LLM speed and memory: Prompt processing · prefill 308–612 tok/s · Token generation · decode 42.8–50.3 tok/s; Required 31.99GB / Usable 116GB unified memory
4,096 input tokens · single request · prompt cache off
Calculate speed with this configurationMethod and evidenceCC BY 4.0

Hardware core specifications

Available memory
116GB
Memory bandwidth
273GB/s
Estimated sustained LLM load
About 190W
Korean price range (KRW)
952 × KRW 10,000–1,030 × KRW 10,000

Who it fits

  • Load models that need more than 32GB
  • CUDA and ARM64 serving development
  • Run on one machine without multiple GPUs

Skip it if

Do not buy a Spark first if the tools you need are x86-only with no ARM64 execution path. Check the containers and extensions you will use on DGX OS. Connecting two units with a cable does not simply pool their memory; you need a setup that splits the model across them.

Recommended format for each device 4K input

The model you will experience with this equipment

K2 Horizon 0.9B

Required 1.03GB · spare time

llama.cpp · Q4_K_M

Decode

370.9 ~ 412.1 tok/s

First token

0.39 to 0.71 seconds

Qwen3.8-Flash-Next

recommended

Required 107.13GB · titration

llama.cpp · Q4_K_M

Decode

16.7 ~ 23.4 tok/s

First token

2.7 to 4.9 seconds

speed tuning

DeepSeek-V4.1-Flash

Required 264.514761248GB · tight

CUDA-cMoE · mmap demand paging · text only

Decode

2.8 tok/s · Published CUDA-cMoE reference

First token

2155.8 s · Not a direct TTFT measurement

speed tuning

Actual speed varies with runtime version, cooling, context length and quantization file.

purchase judgment

What tasks is it suitable for?

Large LLMs

Suitable

Judge fit from the combined size of the model file and input cache. Parameter count alone cannot establish whether it fits, and fitting is a separate question from response speed.

serving development

Suitable

First check that your model is supported by a Spark runtime and ARM64 packages. Copying a command from a desktop setup may not be all that is needed.

Create image

Conditional

Distinguish tasks that need large memory from tasks that need one image quickly. Choose after comparing GeForce with the same model and resolution.

The first thing to get stuck
128GB is the total memory shared by the CPU and GPU. The OS and input cache also need space, and more capacity does not give it the same model-reading bandwidth as GeForce. Compare single-request speed separately from the number of requests handled at once.
Configuration to check
GB10 · 128GB unified memory · One complete system · Check SSD capacity and warranty for the listed configuration
Criteria for spending more money
If you only want one 27B chat session, try your existing PC or a single GPU first. If you need plenty of memory but not necessarily CUDA, the Ryzen AI Max+ 395 128GB is also worth comparing. Consider two Sparks after identifying the capacity or throughput that one unit cannot provide.

On sales pages, look at memory before chips.

The model and speed assumptions above change if the product is not the 128GB configuration.

Frequently asked questions

Which local LLM can be run on DGX Spark 128GB?
K2 Horizon 0.9B, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash and others with a 4K input and the recommended format. Required memory varies by model and context length.
How much local LLM computing power does DGX Spark 128GB have?
Sustained load is about 190 W, with an actual range of 160–233 W.
Are the displayed token speeds ground truth?
This is a range aligned to collected benchmark measurements under the same conditions. Combinations without a matching measurement are estimated from the nearest result and vary with runtime, cooling, quantization file and context length.

Equipment to compare together

CUDA workloads that fit in 32GB

RTX 5090 32GB

Compare this card if you repeatedly generate images or run 27B–35B inference and want shorter waits. But 32GB is still a limit. Buying on compute performance alone, without checking the model, input and image resolution, may leave you offloading again.

When you need 128GB on a Mac

Studio M4 Max 128GB

Suits people who want to compare a wide range from 70B-class to high-precision models in a single finished machine.

128GB without a CUDA requirement

Ryzen AI Max+ 395 128GB

It is attractive when you have large amounts of integrated memory at a low price and can handle runtime settings directly.