Executable programs and extensions
Last updated
How to run Nemotron 3.5 Super VL on DGX Spark or RTX PRO 6000
Choose the runtime recipe and input size before planning around a 120B model name.
Nemotron 3.5 Super VL is a multimodal MoE1 model with 120B total and 12B active parameters. It accepts text, images, and video frames2, then returns text. The published vLLM paths differ: DGX Spark uses LiLiCorr, while one RTX PRO 6000 recipe uses NVFP43 and native MTP4 with 5 draft tokens5. This guide distinguishes their context allocations and input paths.
Super VL takes a prompt6 with text, images, or video frames and returns text. It has 120B total parameters, activates 12B per token, and uses the C-RADIO vision encoder. For local inference7, first choose between the DGX Spark LiLiCorr recipe and the RTX PRO 6000 native-MTP recipe. NVIDIA model card
The model’s maximum context is 1M tokens, but the selected hardware recipe sets the actual allocation. The vLLM recipe lists context 114,688 for DGX Spark NVFP4 and 16,384 for RTX PRO 6000 NVFP4. These are allocations for those cookbook profiles, not a universal limit for every deployment. vLLM recipe table
What can you send and receive?
Super VL is a vision-language model that accepts text, images, and frames sampled from video. You can send a document table or screen image and receive a text response. The model card supports MP4 video up to 2 minutes. Image and video requests use context for visual tokens and frames, so start with a small input.
The model combines Mamba-2, MoE, and attention8 layers with a C-RADIO encoder and vision-language connector. The NVFP4 weight index is 75.01 GB; the vLLM recipe calculates a 91 GB minimum with a 20% allowance. That is a weight-loading baseline: context, batch size9, and media-frame settings use additional memory. Hardware-specific vLLM recipes
DGX Spark path: run with LiLiCorr drafting
To use NVIDIA's vLLM quick start on DGX Spark (GB10), first run the official vllm/vllm-openai:v0.32.0 container or the listed nightly container. Enter the vllm serve command below in a terminal inside the container. It runs the NVFP4 target with a separate LiLiCorr drafter.
The settings are GPU10 utilization 0.75, 8,192 batched tokens, one concurrent sequence, and 7 draft tokens. NVIDIA Super VL model card · Official vLLM container image
Start the server in the container's first shell and wait for model loading and kernel compilation. Leave the server running, open another shell in the same container, and run the check below. The API11 is ready when the model list includes nemotron-3.5-super-vl. Follow the model-card quick-start for the container and its detailed options.
export MODEL=nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4
export DRAFT_MODEL=nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4-LiLiCorr
vllm serve "$MODEL" \
--host 127.0.0.1 \
--served-model-name nemotron-3.5-super-vl \
--max-num-seqs 1 \
--trust-remote-code \
--gpu-memory-utilization 0.75 \
--max-num-batched-tokens 8192 \
--speculative_config.model "$DRAFT_MODEL" \
--speculative_config.num_speculative_tokens 7 \
--speculative_config.method dflash \
--mamba-backend flashinfer \
--mamba-ssm-cache-dtype float16 \
--enable-mamba-cache-stochastic-rounding \
--mamba-cache-philox-rounds 5 \
--enable-mamba-shared-prefix-checkpoint \
--video-pruning-rate 0.5 \
--media-io-kwargs '{"video": {"fps": 2, "num_frames": 256}}' \
--block-size 64 \
--prefix-match-unit 64 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--reasoning-parser nemotron_v3curl --fail http://127.0.0.1:8000/v1/modelsRTX PRO 6000 path: NVFP4 with native MTP 5
The vLLM cookbook provides an NVFP4 profile for one RTX PRO 6000 Blackwell. Its settings are tensor parallel12 1, pipeline parallel13 1, data parallel 1, max model length 16,384, and up to 128 concurrent sequences.
This profile uses a target-native MTP draft of 5. It is a different path from DGX Spark, which specifies a separate LiLiCorr drafter. Single RTX PRO 6000 profile and command
The RTX PRO 6000 profile on the recipe page is labeled for a text workload. For image or video input, check for a profile that explicitly matches the GPU and media mode; a text-only command does not establish multimodal support. Begin with the profile’s sequence and context settings, then test with your own inputs.
vllm serve nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4 \
--host 127.0.0.1 \
--served-model-name nemotron-3.5-super \
--trust-remote-code \
--tensor-parallel-size 1 \
--pipeline-parallel-size 1 \
--data-parallel-size 1 \
--max-model-len 16384 \
--max-num-seqs 128 \
--speculative_config.method mtp \
--speculative_config.num_speculative_tokens 5 \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--reasoning-parser nemotron_v3
Match input type and limits when comparing
The DGX Spark LiLiCorr and RTX PRO 6000 native-MTP profiles here are separate recipes for different hardware, so comparing them also includes differences in device and context allocation. To isolate LiLiCorr versus MTP, measure both on the same hardware and runtime14 while holding target, input, sampling, context, and concurrency constant.
Scroll sideways to compare all columns.
| Hardware | recipe | profile context | Speculative setting |
|---|---|---|---|
| DGX Spark (GB10) | vLLM + NVFP4 | 114,688 | LiLiCorr 7 draft tokens |
| 1× RTX PRO 6000 | vLLM + NVFP4 | 16,384 | native MTP 5 draft tokens |
The 1M figure is the model’s maximum context. Published quick starts and hardware profiles use smaller allocations. Verify an image request with one small PNG, then add document pages. For video, start from the card’s 2 FPS15 and up-to-256-frame settings, adjusting clip length and resolution. Super VL API and media examples
The table’s context and speculative settings are the values listed for each hardware profile in the vLLM cookbook.
Terminology notes
MoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the textFrame — A single image that makes up part of a video.
Back to the textNVFP4 — A 4-bit floating-point data format defined by NVIDIA. Support depends on GPU generation, model, and software implementation.
Back to the textMTP — A training objective or model component for predicting multiple future tokens at once. In speculative decoding, those predictions can be proposed as candidate tokens.
Back to the textToken — A unit into which a model divides input or output for processing.
Back to the textPrompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the textInference — The process of using a trained model to compute an output for an input. Here, local inference means running the model on the user’s device.
Back to the textAttention — A neural-network operation that calculates relationships between input tokens and combines information from each position into the current representation.
Back to the textBatch size — The number of inputs or requests processed together in one batch.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textAPI — A defined interface that lets other code call a program’s functions.
Back to the textTensor parallelism — A method that splits matrices and computations within a model layer across devices for joint execution. Devices communicate to combine partial results.
Back to the textPipeline parallelism — A parallelization method that assigns groups of model layers to device stages and passes each stage’s output activations to the next device.
Back to the textRuntime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textFPS — Frames per second.
Back to the text
Frequently asked questions
- What should you check first in this article?
- Nemotron 3.5 Super VL is a multimodal MoE model with 120B total and 12B active parameters. It accepts text, images, and video frames, then returns text. The published vLLM paths differ: DGX Spark uses LiLiCorr, while one RTX PRO 6000 recipe uses NVFP4 and native MTP with 5 draft tokens. This guide distinguishes their context allocations and input paths.
- How much memory does it need?
- The model combines Mamba-2, MoE, and attention layers with a C-RADIO encoder and vision-language connector. The NVFP4 weight index is 75.01 GB; the vLLM recipe calculates a 91 GB minimum with a 20% allowance. That is a weight-loading baseline: context, batch size, and media-frame settings use additional memory. [Hardware-specific vLLM recipes](https://recipes.vllm.ai/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4)
- When is it better not to choose this?
- The RTX PRO 6000 profile on the recipe page is labeled for a text workload. For image or video input, check for a profile that explicitly matches the GPU and media mode; a text-only command does not establish multimodal support. Begin with the profile’s sequence and context settings, then test with your own inputs.
- Where can you check the conditions and sources?
- The settings are GPU utilization 0.75, 8,192 batched tokens, one concurrent sequence, and 7 draft tokens. [NVIDIA Super VL model card](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4) · [Official vLLM container image](https://hub.docker.com/r/vllm/vllm-openai/)