Running DeepSeek V4.1 Flash locally: the numbers beyond 552B

The 552B figure counts the 40-layer backbone. Two large Engram tables add 196B, while the checkpoint also contains the vision encoder, projector and speculative-decoding components. Treating 552B as the whole model—or predicting the file from a simple 552B-plus-196B sum—does not describe the packaged release accurately.

The 8B prefill and 16B decode figures describe the backbone paths participating in computation for each token. Unselected expert weights still have to exist in storage and memory. Converting 8B or 16B directly into a VRAM requirement is therefore a category error.

Engram is learned conditional memory: token n-grams are hashed into rows of two FP8 tables and fused into the residual stream. Each step reads only a few rows, but the full tables occupy about 189 GiB. It is not disposable prompt-cache data, nor should it be equated with a generic PLE feature.

SGLang's early implementation can place Engram tables in host memory. That frees GPU capacity but requires substantial system memory and suitable CPU–GPU connectivity; it does not remove the rest of the model or its KV cache. The phrase “offload supported” is not evidence that one small GPU is sufficient.

The official release is a mixed-precision checkpoint using FP8, FP4, BF16 and supporting values; its download is about 475.2 GiB. That is close to a storage floor, not a complete hardware requirement. Conversion can temporarily require both source and output files, while execution adds the runtime, workspaces, image processing and context-dependent KV cache.

A third-party Q2_K file has been measured at roughly 246.3 GiB, but that is not official llama.cpp support. The evidence is a short text smoke test on a limited fork; paths beyond 16K, vision, MTP and quality parity remain unfinished. It is too early to call this a minimum recommended memory size or a complete personal deployment.

Immediately after release, the validated paths are dedicated SGLang and vLLM development images or branches on multi-GPU systems such as H200, B300 and GB300. The short install commands on the model page are starting points, not guarantees that every stable release and hardware target supports every feature. Record the exact commit and deployment recipe if you need tools, images or long context.

Both llama.cpp and MLX have documented runs using dedicated forks or custom runtimes, but these results should not be generalized to standard installations. The M3 Ultra 256GB MLX 2-bit run measured 8.8 tok/s with a 16-token input over 23 timed decode steps, with a runtime context limit of 128 tokens. The RTX 5090 run measured 5.1 tok/s on new inputs using native mixed FP4/FP8 streaming with 128GB RAM and PCIe 5 NVMe. Prefill was not measured for either path.

There is more than one Spark path. A single-Spark CUDA-cMoE experiment that reads Q2_K through CPU mmap reported 1.9 tok/s prefill and 2.8 tok/s decode. The three-Spark configuration using native mixed FP4/FP8 and a dedicated distributed runtime published only a 37.9 tok/s decode result; the four-Spark configuration published 43.12 tok/s decode plus separate cold-prefill anchors. We do not invent TTFT for paths without prefill measurements or transfer figures across different node counts and formats.

DSpark is speculative decoding that uses the checkpoint's multi-step prediction components to propose and verify several tokens. Early SGLang measurements used four B300 GPUs, a specific development branch and a fixed, repetitive weather-notes prompt. That is useful evidence that the feature executes, not a transferable speed promise for one consumer GPU or a different workload.

Combination limits remain. SGLang's encoder SWA bounded replay cannot be used with speculative decoding, and prefill/decode disaggregation with DSpark is still in progress. vLLM has also had a reported CPU KV-offload failure with DSpark enabled. For a purchase, wait for a public test that simultaneously passes the offload, context and image conditions you need instead of projecting a best-case speed.