new model · 2026.09.12
Last updated
Running Ling-3.0-flash-VL locally: what the 70.87 GiB INT4 build means
Read images and video frames with Ling-3.0-flash-VL
Ling-3.0-flash-VL is an approximately 124.85B MoE1 model that reads images and video frames2 and answers in text. SGLang's 5.1B figure is the amount activated per token3, not total weight size. Read the official BF164, INT4, and FP4 files and Spark measurements in their respective conditions.
To find a logo in a product video or ask about a box's color and position, provide relevant frames and receive a text answer. Start with a short test using only the frames needed for the question. This is different from automatically processing an entire video or generating a new one.
Total 124.85B and active 5.1B describe different scales
Total parameters are 124.85B; SGLang's 5.1B refers to the portion active per token. MoE selects experts for computation but needs access to all expert weights. Budget storage and runtime5 memory using the full distribution.
The SGLang configuration defaults to 128K context; 256K uses a YaRN extension. Long documents, video frames, questions, runtime buffers, and KV cache6 share memory. The context limit is a supported range, not a promise that every request processes the same volume, so start with the frame count and text length you actually need.
Choose quantized files and a compatible runtime together
Official weight files are about 232.55 GiB in BF16, 70.87 GiB in INT4, and 64.81 GiB in FP4. These are download and storage sizes; runtime headroom and input-processing memory are additional. If converting a file, allow space for both the source and converted copy.
Official FP4 does not mean NVIDIA NVFP47 or GGUF8, and INT4 is not compatible with every 4-bit loader. Scaling methods and kernels may differ, so confirm that the runtime supports the checkpoint9 format and model architecture. If a file fails to open, check format and kernel support before concluding memory is insufficient.
Video input depends on frame selection and verification
Ling-3.0-flash-VL answers in text after viewing images or video frames; it does not generate video. To ask when a logo appears in a product video, start with a few key frames and a specific question. Setting a 256K context does not automatically submit the full video; frame count, resolution, and description text all use memory.
Open the original frame at the time cited in the answer and verify that the object or action is visible. Distinguish whether the model read a product name in subtitles or actually saw the logo. If you extract frames separately, record their original timestamps in filenames or inputs so another reviewer can find the same scene.
Compare DGX Spark speed within its specified text conditions
The official SGLang guide compares INT4 and MXFP4 on one DGX Spark. The test used an 8,192-token input, 1,024 output tokens, and one concurrent request. Median inter-token latency was 44.18 ms for INT4 and 30.18 ms for MXFP4.
Generation speed was about 22.6 tok/s and 33.1 tok/s, respectively. MXFP4 is a different format from NVFP4, so the two names do not refer to the same setting.
Median time to first token10 was about 2.89 seconds for INT4 and 2.54 seconds for MXFP4. That wait includes input prefill11, while later token intervals describe decode12 speed. The test used text input and excludes image preprocessing; it cannot be generalized to other GPUs or Macs. A one-request measurement also cannot establish capacity for multiple team members.
Test video tasks separately with short clips whose answers you know. If the model says the logo is on the right side of the box around 00:12, compare that with the original frame and check whether it read a subtitle or saw the logo itself. Keep runtime format, prompt13, output budget, and input-token count constant, and record time to first token separately from generation.
Scroll sideways to compare all columns.
| Format | Median inter-token latency | Converted generation speed | Median time to first token |
|---|---|---|---|
| INT4 | 44.18ms | about 22.6 tok/s | about 2.89 s |
| MXFP4 | 30.18ms | about 33.1 tok/s | about 2.54 s |
Terminology notes
MoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the textFrame — A single image that makes up part of a video.
Back to the textToken — A unit into which a model divides input or output for processing.
Back to the textBF16 — A 16-bit floating-point format for model values. Its exponent has the same width as FP32, giving it a similarly wide value range.
Back to the textRuntime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textKV cache — Memory that stores attention keys and values from earlier tokens for reuse during later token generation. Its size depends on context length and batch size.
Back to the textNVFP4 — A 4-bit floating-point data format defined by NVIDIA. Support depends on GPU generation, model, and software implementation.
Back to the textGGUF — A file format for model data, widely used by llama.cpp-based tools.
Back to the textCheckpoint — A file containing saved model weights and related state. Versions or tasks in one model family may use different checkpoints.
Back to the textTime to First Token — The time from sending a request until its first output token arrives.
Back to the textPrefill — The stage where an LLM reads the input prompt and computes representations for its tokens. Longer prompts contain more tokens to process.
Back to the textToken decoding — The stage that generates successive output tokens using the input and the response produced so far.
Back to the textPrompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the text
Frequently asked questions
- What should you check first in this article?
- Ling-3.0-flash-VL is an approximately 124.85B MoE model that reads images and video frames and answers in text. SGLang's 5.1B figure is the amount activated per token, not total weight size. Read the official BF16, INT4, and FP4 files and Spark measurements in their respective conditions.
- How much memory does it need?
- Total parameters are 124.85B; SGLang's 5.1B refers to the portion active per token. MoE selects experts for computation but needs access to all expert weights. Budget storage and runtime memory using the full distribution.
Change log
These entries record changes to the site's guidance. They do not automatically check your installed engine or model version.
Single-Spark execution paths for new models
The speed experience now uses published single-Spark measurements for DeepSeek and Ling. The same speeds and commands are not applied to unverified hardware.