new model · 2026.09.12
Last updated
Running DeepSeek V4.1 Flash locally: the numbers beyond 552B
Check the full checkpoint and runtime before active parameters
DeepSeek V4.1 Flash is a large MoE1 model generating answers from documents and questions. Its official checkpoint2 is roughly 475.2GiB and includes backbone, Engram tables, and speculative-decoding3 components. This guide covers reported personal-machine streaming experiments through multi-Spark configurations.
If you want to review a set of contracts on a Mac, check not only whether the files fit on the SSD4 but whether the runtime5 can place all components in memory. Storage, system and GPU6 placement, and the exact runtime branch come before public scores.
Backbone, Engram, and active parameters describe different scales
DeepSeek V4.1 Flash has a 40-layer 552B backbone plus 196B Engram. The stated 8B input and 16B generation describe paths activated during token7 computation. Unselected expert weights still reside in storage and memory, so 16B active does not translate to 16 GB VRAM8.
The official mixed-precision checkpoint combines FP89, FP4, and BF1610 components and is about 475.2 GiB. It also contains a vision encoder/projector and speculative-decoding components. Plan separately for SSD storage and memory for the backbone, runtime buffers, and KV cache11.
Engram is learned memory that hashes token n-grams, retrieves rows from FP8 tables, and combines them with the residual stream. Even if one computation reads only selected rows, the complete tables occupy about 189 GiB. This is not temporary prompt12 cache or ordinary PLE13, and a short input does not shrink the resident table to match context length14.
A 1017 update has arrived
A 1017 checkpoint for the same slot was released on October 17, 2026. The architecture and file sizes are unchanged, but all 48 shards were rewritten, including Engram and vision files. The 475.25 GiB and runtime conditions in this guide still apply to 1017, but download into a new folder instead of overwriting the files you already have.
The 1017 update guide continues with what changed and which runtimes read it.
Budget offloaded memory and remaining space
The early SGLang implementation can move Engram tables into host memory. That reduces GPU residency but requires substantial system RAM15 and transfers across the CPU-GPU interconnect. Backbone weights and context-dependent KV cache remain. Do not assume that a 256 GB unified-memory system has ample space merely because offloading16 exists; check where every component is placed.
A third-party Q2_K file of about 246.3 GiB has also been published, but the verified evidence is a short-text smoke test on a limited fork. A smaller file is not evidence of reliable long-document handling.
Read published runtime measurements with their hardware and scope
Personal-hardware reports range from short tests to multi-Spark servers, with different setups and inputs. A user MLX17 2-bit path on M3 Ultra 256 GB reported 8.8 tok/s over 23 decode18 steps after a 16-token input, with a 128-token context cap. An RTX 5090 setup with 128 GB RAM and PCIe 5 NVMe19 streamed the original mixed FP4/FP8 weights and reported 5.1 tok/s on a new input. Neither report measured prefill20, so time to first read a long contract set is unknown.
Spark results are not the same test either. A single-Spark CUDA-cmoe run reading Q2_K through CPU21 mmap reported 1.9 tok/s prefill and 2.8 tok/s decode. A three-Spark distributed-runtime setup with original mixed FP4/FP8 weights reported 37.9 tok/s decode; a four-Spark setup reported 43.12 tok/s decode plus a separate cold-prefill reference. Do not infer time to first token22 from measurements that omit prefill.
| Setup | Input and runtime conditions | Published result |
|---|---|---|
| M3 Ultra 256 GB, user MLX 2-bit | 16-token input, 23 decode steps, 128-token context cap | 8.8 tok/s; prefill not measured |
| RTX 5090, RAM 128GB, PCIe 5 NVMe | Original mixed FP4/FP8 streaming, new input | 5.1 tok/s; prefill not measured |
| 1 Spark, CUDA-cmoe, Q2_K CPU mmap | Single-system test | prefill 1.9, decode 2.8 tok/s |
| 3 Spark systems, original mixed FP4/FP8 distributed | Distributed runtime | decode 37.9 tok/s |
| 4 Spark systems | Separate cold-prefill reference | decode 43.12 tok/s |
Development runtimes and speculative decoding still have limits
The main paths immediately after release were dedicated development images or branches for SGLang and vLLM, targeting multi-GPU servers such as H200, B300, and GB300 systems. Model-page installation commands are starting guidance, not proof that tool calls23, images, and long contexts work on every stable release and device. Record the exact development commit and deployment recipe, then test each capability.
DSpark is speculative decoding: the checkpoint's multi-token prediction24 component proposes candidates that the main model verifies. Initial SGLang results show the feature working on four B300 GPUs, a specific development branch, and a fixed repeating weather-note input. They are not measurements for other prompts or a single consumer GPU.
Combination limits also matter. SGLang's encoder SWA bounded replay cannot be used with speculative decoding, and prefill/decode disaggregation with DSpark integration is in progress. An error has been reported for CPU KV offloading with DSpark enabled in vLLM. Choose hardware only after the required offloading, context, and image input work together in one configuration.
Terminology notes
MoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the textCheckpoint — A file containing saved model weights and related state. Versions or tasks in one model family may use different checkpoints.
Back to the textSpeculative decoding — A generation method in which a draft model, MTP head, or repeated-context lookup proposes output candidates for the main model to verify and accept.
Back to the textSSD — A data storage device that uses flash memory and retains data without power.
Back to the textRuntime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textToken — A unit into which a model divides input or output for processing.
Back to the textVRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the textFP8 — A floating-point format that represents each value with 8 bits.
Back to the textBF16 — A 16-bit floating-point format for model values. Its exponent has the same width as FP32, giving it a similarly wide value range.
Back to the textKV cache — Memory that stores attention keys and values from earlier tokens for reuse during later token generation. Its size depends on context length and batch size.
Back to the textPrompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the textPLE embedding table — Short for Per-Layer Embedding, a structure that adds embedding information at particular model layers. The Qwen Flash-Next family uses n-gram tables; supported runtimes can offload these tables to RAM or SSD.
Back to the textContext window — The token span of input and generated content a model can handle in one request. The supported limit and memory use depend on the model and runtime settings.
Back to the textSystem RAM — System memory that temporarily holds data while programs run.
Back to the textOffloading — Moving some model data from GPU memory to system RAM or storage when capacity is limited. This adds data transfer.
Back to the textMLX — Apple’s machine-learning framework, which runs models using memory shared by the CPU and GPU on Apple Silicon.
Back to the textToken decoding — The stage that generates successive output tokens using the input and the response produced so far.
Back to the textNVMe — A storage protocol used by SSDs communicating over PCI Express.
Back to the textPrefill — The stage where an LLM reads the input prompt and computes representations for its tokens. Longer prompts contain more tokens to process.
Back to the textCPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.
Back to the textTime to First Token — The time from sending a request until its first output token arrives.
Back to the textTool call — A structured request from a model for an external function such as reading a file, searching, or running a command. The agent runtime and its permission settings decide whether the request is actually executed.
Back to the textMTP — A training objective or model component for predicting multiple future tokens at once. In speculative decoding, those predictions can be proposed as candidate tokens.
Back to the text
Frequently asked questions
- What should you check first in this article?
- DeepSeek V4.1 Flash is a large MoE model generating answers from documents and questions. Its official checkpoint is roughly 475.2GiB and includes backbone, Engram tables, and speculative-decoding components. This guide covers reported personal-machine streaming experiments through multi-Spark configurations.
- How much memory does it need?
- DeepSeek V4.1 Flash has a 40-layer 552B backbone plus 196B Engram. The stated 8B input and 16B generation describe paths activated during token computation. Unselected expert weights still reside in storage and memory, so 16B active does not translate to 16 GB VRAM.
Change log
These entries record changes to the site's guidance. They do not automatically check your installed engine or model version.
Single-Spark execution paths for new models
The speed experience now uses published single-Spark measurements for DeepSeek and Ling. The same speeds and commands are not applied to unverified hardware.