Image and video generation

Turn an image into a short video with Cosmos 3 Edge on DGX Spark

Feed it a still image and JSON prompt; Cosmos 3 Edge returns a generated MP4.

Cosmos 3 Edge's Generator turns a reference image and a video-oriented JSON prompt1 into a short clip. This guide starts the ARM64-capable vLLM-Omni container on DGX Spark, sends the published I2V sample image to a local API2, and saves the returned MP4. The model card's DGX Spark timings are published results for 189 frames3; read them as separate runtime4 paths. Text reasoning and action prediction belong to different tasks.

Requirements and key details
  • Cosmos 3 Edge's Generator creates an MP4 from a reference image and a video JSON prompt.
  • On DGX Spark, run the vLLM-Omni container as ARM64 and bind its API to a local port.
  • Published I2V timings use 480p and 189 frames; they are separate from Reasoner results and other GPUs.

It turns an image into a short MP4

Cosmos 3 Edge is NVIDIA's world model for generating or interpreting physical-world content from text, images, and actions. This guide follows one Generator path: image-to-video (I2V). Send a still image and a structured JSON prompt describing the continuation; the API returns an MP4. Reasoner requests for action prediction or video understanding are separate workloads and should not be compared with these video-generation times. Cosmos 3 Edge model card

DGX Spark is an ARM64 Linux system built around GB10. The official vLLM-Omni Cosmos3 image manifest includes linux/arm64, so this recipe selects that platform. The command binds the API only to 127.0.0.1; the request uploads the image directly. vLLM-Omni Cosmos3-Edge recipe

A compact computer sits beside a monitor showing code and a chat window.
A local model server turns the request into a saved video file.

Start the ARM64 server on a local Docker port

Prepare the NVIDIA driver and NVIDIA Container Toolkit on DGX Spark. For the default guardrails, first read the terms on the Cosmos Guardrail model page and obtain access for your own account. Then use the commands below to create a Python environment, sign in, and start the server. It is ready when the logs show Application startup complete. DGX Spark User Guide

Run the Cosmos 3 Edge server locally
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U huggingface_hub requests
hf auth login
cosmos_cache="${HF_HOME:-$HOME/.cache/huggingface}"
mkdir -p "$cosmos_cache"
docker run --rm --name cosmos3-edge --platform linux/arm64 \
  --runtime nvidia --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v "$cosmos_cache:/root/.cache/huggingface" \
  vllm/vllm-omni:cosmos3 \
  vllm serve nvidia/Cosmos3-Edge --omni --host 0.0.0.0 --port 8000 \
    --init-timeout 1800

Leave the server running and open a second terminal in the same working directory. Reactivate the environment and download only the sample image and JSON prompts. Inspecting the output video requires ffprobe, included with FFmpeg.

Download the official I2V example inputs
source .venv/bin/activate
hf download nvidia/Cosmos3-Edge --include "assets/*" --local-dir Cosmos3-Edge

Upload the sample image and structured prompt

Move into the downloaded sample folder and run the Python client below. It reads the model card's JSON prompt and negative prompt, then uploads the image as a multipart request to the local server. The request asks for 121 frames at 832×480 and 24 fps5; it writes the returned bytes to cosmos3_edge_i2v.mp4 in the current directory. I2V request format in the model card

Send the I2V request and inspect the MP4
cd Cosmos3-Edge
python - <<'PY'
import json, mimetypes, requests
from pathlib import Path
prompt = json.load(open('assets/example_i2v_prompt.json'))
negative = json.load(open('assets/negative_prompt.json'))
image = Path('assets/example_i2v_input.jpg')
data = {
    'prompt': json.dumps(prompt),
    'negative_prompt': json.dumps(negative),
    'size': '832x480',
    'num_frames': '121',
    'fps': '24',
    'num_inference_steps': '20',
    'guidance_scale': '6.0',
    'max_sequence_length': '4096',
    'flow_shift': '12.0',
    'seed': '0',
    'extra_params': json.dumps({
        'use_resolution_template': False,
        'use_duration_template': False,
        'guardrails': True,
    }),
}
mime = mimetypes.guess_type(image.name)[0] or 'image/jpeg'
with image.open('rb') as file:
    response = requests.post('http://127.0.0.1:8000/v1/videos/sync', data=data, files={'input_reference': (image.name, file, mime)}, headers={'Accept': 'video/mp4'}, timeout=1800)
response.raise_for_status()
output = Path('cosmos3_edge_i2v.mp4')
output.write_bytes(response.content)
print(f'Saved {output} ({len(response.content):,} bytes)')
PY
ffprobe -v error -select_streams v:0 -show_entries stream=width,height,nb_frames,r_frame_rate -of default=noprint_wrappers=1 cosmos3_edge_i2v.mp4

The server returns the video bytes and the client saves them. Afterward, use ffprobe to inspect the video dimensions and frame count and confirm the request produced an MP4 file.

A cyclist on a coastal road appears in one still image and several moving-scene frames.
The reference image anchors the opening frame; the prompt describes what follows.

Read the published generator timings as separate records

On DGX Spark, the runtime affects how long the same image-to-video task takes. The table reports published generation of 189 frames. The shorter example above requests 121 frames, so keep the frame count alongside each timing.

DGX Spark Generator image-to-video records in the model card
RuntimeDGX Spark I2V latency
vLLM-Omni165.96 s
PyTorch179.80 s

DGX Spark · Generator I2V · 480p · 189 frames · one integrated platform · batch size6 1. Conditions and results

If the server will not start, check docker info for the NVIDIA runtime. For a 422 response, inspect the server log and request fields, then confirm the JPEG and official JSON files are under Cosmos3-Edge/assets/. An MP4 stream reported by ffprobe confirms the output file was written.

Several frames of a cyclist on a mountain road run across a strip of film.
Video output is a sequence of frames, so frame count and size affect generation time.

Terminology notes

  1. Prompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.

    Back to the text
  2. API — A defined interface that lets other code call a program’s functions.

    Back to the text
  3. Frame — A single image that makes up part of a video.

    Back to the text
  4. Runtime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.

    Back to the text
  5. FPS — Frames per second.

    Back to the text
  6. Batch size — The number of inputs or requests processed together in one batch.

    Back to the text

Frequently asked questions

What should you check first in this article?
Cosmos 3 Edge's Generator turns a reference image and a video-oriented JSON prompt into a short clip. This guide starts the ARM64-capable vLLM-Omni container on DGX Spark, sends the published I2V sample image to a local API, and saves the returned MP4. The model card's DGX Spark timings are published results for 189 frames; read them as separate runtime paths. Text reasoning and action prediction belong to different tasks.
Where can you check the conditions and sources?
Cosmos 3 Edge is NVIDIA's world model for generating or interpreting physical-world content from text, images, and actions. This guide follows one Generator path: image-to-video (I2V). Send a still image and a structured JSON prompt describing the continuation; the API returns an MP4. Reasoner requests for action prediction or video understanding are separate workloads and should not be compared with these video-generation times. [Cosmos 3 Edge model card](https://huggingface.co/nvidia/Cosmos3-Edge)