Image and video generation

Run the Matrix-Game 4.0 Local Browser Demo on an RTX 5090

The game engine calculates input and physics while the video model renders the view.

Create a Python 3.12 environment with the CUDA¹ 13.2 PyTorch² packages in the Matrix-Game-4 repository and select the rtx5090 preset. Download the Ancient Egypt model and game assets from manifest revision 2c53100457da7abcf0e966f75637c644d3423a63, build the native game, then run MediaMTX and the model server. The local demo opens at http://127.0.0.1:8025/.

Contents

Requirements and key details

Run the inference server on Linux with Python 3.12 and an NVIDIA GPU.
The 5090 preset is --preset rtx5090 and does not require a separate quantized checkpoint.
--fps 40 is the game capture target cadence, not a model-throughput or latency measurement.
Open http://127.0.0.1:8025/ on the server PC to reach the demo.

The engine runs the game, and the model renders its view

Matrix-Game 4.0 runs an executable low-poly game: its game engine handles keyboard input and physics, while a causal video model streams clean RGB views. The HUD and sound are synchronized outside the renderer. This installation path uses one RTX 5090 to control the Ancient Egypt scene from a local browser. Official v4 README

A monitor shows two mountain-road views beside a desktop computer and film reel — The runtime needs both the game client and the inference server.

Prepare the Linux and RTX 5090 runtime

Inference³ requires Linux, an NVIDIA GPU⁴, and Python 3.12. The same host that runs the browser demo also needs the native game and GStreamer; follow the official installation guide for system dependencies such as CMake, C++20, SDL2, and FFmpeg. A model-only test and the browser demo do not require identical setup.

The RTX 5090 path explicitly selects rtx5090 or the optional rtx5090-sage preset. The standard rtx5090 preset uses PyTorch SDPA and does not require SageAttention. The runtime⁵ does not automatically select a matching preset for other GPUs. The current README publishes the optimized path for the 5090 and lists 5080 and 5070 support as a later release. RTX 5090 setup

Install PyTorch and Matrix-Game in a virtual environment

Clone the repository and enter its v4 folder. These commands create a Python 3.12 virtual environment⁶, then install PyTorch 2.12.0 for CUDA 13.2 and the Matrix-Game inference, server, and download dependencies. Run them from the repository’s Matrix-Game-4 directory.

Create the Python environment for an RTX 5090

git clone https://github.com/SkyworkAI/Matrix-Game.git
cd Matrix-Game/Matrix-Game-4
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install torch==2.12.0 torchvision==0.27.0 --index-url https://download.pytorch.org/whl/cu132
python -m pip install -e '.[inference,server,download,test]'

Download Ancient Egypt files from the pinned revision

Download the game data and model separately. Using the same official Hub revision, 2c53100457da7abcf0e966f75637c644d3423a63, for both manifests keeps the asset list and checkpoint⁷ aligned with the code. The example saves the Ancient Egypt scene under resources/. Keep the downloaded files in this directory rather than moving them into Git-tracked source paths.

Download the scene assets and checkpoint

matrix-game download --manifest releases/game-assets.json \
  --repo-id Skywork/Matrix-Game-4.0 --repo-type model \
  --revision 2c53100457da7abcf0e966f75637c644d3423a63 \
  --scene ancient-egypt --resources resources

matrix-game download --manifest releases/models.json \
  --repo-id Skywork/Matrix-Game-4.0 --repo-type model \
  --revision 2c53100457da7abcf0e966f75637c644d3423a63 \
  --scene ancient-egypt --resources resources

A film strip shows several consecutive scenes of a cyclist moving along a road — Control input changes the game state, and the model renders that state.

Build the game and start both services

After installing the native libraries and browser-demo media dependencies listed in the setup guide, build the game from Matrix-Game-4/. MediaMTX and the model server run in separate terminals, so leave each process open while you start the next one in another window. Browser demo requirements

Build the game

bash scripts/build_game.sh

Terminal 1: start MediaMTX

mediamtx configs/mediamtx.yml

Terminal 2: start the RTX 5090 inference server

CUDA_VISIBLE_DEVICES=0 bash scripts/serve.sh \
  --scene ancient-egypt --resources resources --preset rtx5090 \
  --prompt-mode fixed --fps 40

Open the scene in a browser and isolate the first error

When both processes are running, open http://127.0.0.1:8025/ on the server PC. Use the controls and check that the scene and HUD appear. The --fps 40 option sets the target cadence for capturing game frames⁸. It does not measure 40 model-generated frames per second⁹ or a 1/40-second response after a key press. Browser decoding¹⁰, transport, and buffering also affect the observed response.

If the page does not open, first check that the MediaMTX and model-server terminals are still running. If the server reports missing model files, run matrix-game verify from the repository root for both manifests to locate them. If CUDA initialization fails, check that this virtual environment has the CUDA 13.2 PyTorch build and that the NVIDIA driver supports the combination. Verification checks installed files and checksums without downloading the models again.

The browser demo is for a trusted single user, not an authenticated public service. HTTP, RTSP, and WebRTC bind to loopback by default, so do not expose them to the Internet by forwarding ports or changing firewall rules. If you need remote access, follow the documented SSH-forwarding procedure and security notes. Official security guide

A studio with monitors showing video scenes and computers of different sizes — The computer illustration is not a screenshot of Matrix-Game 4.0.

Terminology notes

CUDA — A software platform for general-purpose computing on NVIDIA GPUs.
Back to the text
PyTorch — A software framework for building and running AI models. Check the compatible PyTorch version and hardware support along with the model.
Back to the text
Inference — The process of using a trained model to compute an output for an input. Here, local inference means running the model on the user’s device.
Back to the text
GPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the text
Runtime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the text
Python virtual environment — An isolated space for installing Python packages per project, helping reduce package-version conflicts.
Back to the text
Checkpoint — A file containing saved model weights and related state. Versions or tasks in one model family may use different checkpoints.
Back to the text
Frame — A single image that makes up part of a video.
Back to the text
FPS — Frames per second.
Back to the text
Token decoding — The stage that generates successive output tokens using the input and the response produced so far.
Back to the text

Frequently asked questions

What should you check first in this article?: Create a Python 3.12 environment with the CUDA 13.2 PyTorch packages in the Matrix-Game-4 repository and select the `rtx5090` preset. Download the Ancient Egypt model and game assets from manifest revision `2c53100457da7abcf0e966f75637c644d3423a63`, build the native game, then run MediaMTX and the model server. The local demo opens at `http://127.0.0.1:8025/`.
When is it better not to choose this?: The browser demo is for a trusted single user, not an authenticated public service. HTTP, RTSP, and WebRTC bind to loopback by default, so do not expose them to the Internet by forwarding ports or changing firewall rules. If you need remote access, follow the documented SSH-forwarding procedure and security notes. [Official security guide](https://github.com/SkyworkAI/Matrix-Game/blob/main/Matrix-Game-4/SECURITY.md)