How much memory does it need?

MoE models such as Qwen3.6 select a subset of expert weights for each token. PolyStrata caches frequently used experts in GPU memory and computes the rest on the CPU from system RAM. This lets a model file larger than 8GB run across GPU and system memory.

Executable programs and extensions

Can an 8GB GPU run Qwen and Gemma faster? Install PolyStrata and use two GPUs

PolyStrata uses your GPU and CPU together to run large MoE models on your PC.

With an 8GB graphics card and 32GB of system RAM¹, you can try running Qwen3.6 or Gemma 4. Choose a model size that fits your RAM, install it, and send a prompt² in the browser.

Contents

Requirements and key details

On the 8GB laptop, Qwen3.6 UD-Q3_K_M generated 94–100 tok/s in short chats.
Model files live on the SSD; system RAM and the CPU handle expert computation that does not fit on the GPU.
Two NVIDIA GPUs split the model layers on Linux. A two-GPU Windows setup was not tested.
After installation, you can use local browser chat or connect an OpenAI-compatible app.

Keep frequently used experts in GPU memory

MoE³ models such as Qwen3.6 select a subset of expert weights for each token⁴. PolyStrata caches frequently used experts in GPU⁵ memory and computes the rest on the CPU⁶ from system RAM. This lets a model file larger than 8GB run across GPU and system memory.

You can also run Qwen3.8-Flash-Next, using the unchanged Strata engine. Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder, and GLM 5.3 Flash use PolyStrata's dedicated engine. You can switch between installed models while the server is running and connect them to local chat or coding agents.

A black tower PC, a blank monitor, and RAM modules on a warm wood desk indoors — Open browser chat on the PC running the model.

Pair an 8GB GPU with enough system RAM

You can use NVIDIA RTX 20-series or newer cards with at least 8GB VRAM⁷; the published 8GB runs used a laptop with 32GB RAM. Because weights that do not fit on the GPU stay in RAM, you need both SSD⁸ space for the files and enough system RAM to run them. Start by comparing file sizes and RAM requirements for Qwen3.6, Gemma 4, Ornith, and Tiel Coder.

Model files and RAM alongside an 8GB GPU
Model	File size	System RAM
Qwen3.6-35B-A3B	13 / 17 / 23GB	17 / 21 / 27GB
Gemma 4 26B-A4B	11 / 13 / 17GB	15 / 18 / 21GB
Ornith 1.5	14 / 17 / 24GB	18 / 22 / 28GB
Tiel Coder 35B-A3B	14 / 17 / 23GB	18 / 21 / 27GB

Installer RAM estimates alongside an 8GB GPU. The three file sizes and RAM values correspond in the same left-to-right order within each row. Image input adds about 1GB of disk space.

Qwen3.6's middle size uses an approximately 17GB model file and about 21GB of system RAM alongside the GPU. Even on a 32GB PC, close other apps to leave room. For long documents or chat histories, start with a shorter context and increase it gradually.

On Windows, these four models use NVIDIA cards. Linux also offers an AMD beta; the published RX 7600 tests cover Qwen3.6 and Ornith. The AMD beta builds its engine on your PC. AMD support for Qwen3.8-Flash-Next follows the existing Strata path.

GLM 5.3 Flash needs substantially more memory. Its model files are 79–114GB, with roughly 60–96GB or more system RAM alongside a 32GB GPU, depending on the variant. For an 8GB laptop, start with the four models in the table.

A graphics card, RAM modules, and an NVMe SSD on a warm wood desk — VRAM and system RAM handle different parts of model execution.

How fast are Qwen and Gemma on the same 8GB laptop?

The table shows answer-generation speeds in short chats. Changing the quantized file changes speed even within the same model. Results for the RTX 4070 Laptop and RTX 5090 are separated by file.

Generation speed by Qwen3.6 and Gemma 4 file
Model and quantization	RTX 4070 Laptop 8GB	RTX 5090 32GB
Qwen3.6 · UD-Q2_K_XL	123–124 tok/s	450 tok/s
Qwen3.6 · UD-Q3_K_M	94–100 tok/s	394–395 tok/s
Qwen3.6 · UD-Q4_K_M	82–86 tok/s	363–401 tok/s
Gemma 4 · UD-Q2_K_XL	101–104 tok/s	327–340 tok/s
Gemma 4 · UD-Q3_K_M	92–93 tok/s	353–356 tok/s
Gemma 4 · UD-Q4_K_M	85 tok/s	335–346 tok/s

Project-published measurements. Laptop: RTX 4070 Laptop 8GB / Core i7-14700HX / 32GB RAM. Workstation: RTX 5090 32GB / Xeon Platinum 8470Q, 25 vCPU / 120GB RAM. Token-generation speeds in short chats.

On the 8GB laptop, Qwen3.6 generated 94–100 tok/s with the Q3 file and 82–86 tok/s with Q4. Start with Q3 on your usual prompts or code edits; if the answers are unsatisfactory, compare the same request with Q4. Try Q2 when you need more speed or lower RAM use.

Compare long-prompt processing and two-GPU results

Long-document summaries also depend on prompt-reading speed. The table collects generation and prompt-processing results for Qwen3.6 and Gemma 4. It includes single-GPU RTX 4070 Laptop and RTX 5090 results, plus Linux layer splitting across two RTX 3080 cards, each with its own model layers and expert cache.

Scroll sideways to compare all columns.

Model speeds and hardware conditions reported by PolyStrata
Model and configuration	Hardware and input	Generation	Prompt processing
Qwen3.6 UD-Q3_K_M · 1 GPU	RTX 4070 Laptop 8GB, i7-14700HX, 32GB RAM · short chat	94–100 tok/s	30–32K prompt: 1,690 tok/s
Qwen3.6 UD-Q3_K_M · 1 GPU	RTX 5090 32GB, Xeon 8470Q 25 vCPU, 120GB RAM · short chat	394–395 tok/s	30–32K prompt: 6,350–6,550 tok/s
Qwen3.6 UD-Q3_K_M · 2× RTX 3080 10GB	Xeon 8255C · 2,727-token input	2 GPU: 217–221 tok/s / 1 GPU: 165–177 tok/s	2 GPU: 3,425–3,562 tok/s / 1 GPU: 1,567–1,582 tok/s
Gemma 4 UD-Q3_K_M · 2× RTX 3080 10GB	Xeon 8255C · 3,527-token input	2 GPU: 191–193 tok/s / 1 GPU: 182–187 tok/s	2 GPU: 3,540–3,590 tok/s / 1 GPU: 2,170 tok/s

RTX 3080 comparison: 2×10GB at 320W each / Xeon Platinum 8255C / Linux / CUDA⁹ 12.4 / engine dated 2026-10-11. 256 tokens generated after a short prompt; three restarts per configuration for Qwen3.6, two for Gemma 4. Uncached inputs: Qwen3.6 2,727 tokens; Gemma 4 3,527 tokens.

Sources: project measurements in the model README and MULTI_GPU.md.

For the same Qwen3.6 UD-Q3_K_M file with a 30–32K-token input, the RTX 4070 Laptop 8GB processed 1,690 tok/s and the RTX 5090 32GB processed 6,350–6,550 tok/s. In the separate Linux dual-RTX-3080 experiment, Qwen3.6 improved both generation and input processing over one card, while Gemma 4 showed a smaller generation gain. If your Linux desktop has two cards, try layer splitting with your usual model.

Two graphics cards and other components installed in an open PC case — With layer splitting, the two cards compute different layers of the model.

The setup tool guides GPU and model configuration

If GitHub is unavailable, download the source ZIP from the developer's backup¹⁰ page. The backup notice identifies it as version 0.1.43.1-poly.2. Extract the ZIP, or clone the backup repository with the Git commands below.

Install Git and Git LFS before using the Git route. The backup repository uses its source branch and stores some binaries with Git LFS. Skip these commands if you downloaded the ZIP.

Initialize Git LFS and clone the developer backup

git lfs install
git clone -b source https://huggingface.co/spaces/VecSzn/PolyStrata
cd PolyStrata

Run the setup file for your operating system inside the PolyStrata folder: START-HERE.bat on Windows or setup.sh on Linux.

Start installation on Windows

START-HERE.bat

Start installation on Linux

./setup.sh

The setup tool detects the GPU and asks you to choose a model, quantization¹¹, context length¹², and image input support. Choosing the recommended defaults prepares the engine and model. Qwen3.8-Flash-Next is about 70GB and the other models range from roughly 11 to 114GB, so leave enough disk space.

Building the NVIDIA engine locally on Windows requires Visual Studio build tools and the CUDA Toolkit. If a prebuilt engine is unavailable, setup offers to install the tools for a local build. Linux builds require g++ and the CUDA Toolkit.

To use two NVIDIA GPUs on Linux, check their numbers and select both during setup. The tool saves the selection, uses the first card as the primary GPU, and starts layer splitting in auto mode.

Select two NVIDIA GPUs on Linux

nvidia-smi
./setup.sh --gpus 0,1

To use an existing CUDA 12.x toolkit on Linux, specify the option below. The published dual-RTX-3080 measurements used an engine built with CUDA 12.4. Two-GPU operation of PolyStrata's dedicated engine was tested on Linux with NVIDIA cards.

Compile strata-poly with CUDA 12

./setup.sh --cuda 12

Send a first prompt in the browser and check the response

After installation, keep the terminal or batch window open while the model loads. Reading the model into RAM and preparing GPU memory can temporarily slow the PC. Once loading finishes, the window shows the browser address.

Open http://127.0.0.1:8080 in a browser and enter this in the Chat box: “Summarize this in one sentence: Today's meeting starts at 3 p.m.; we will decide whether to deploy after checking QA results.” A summary should appear while the server window remains open. That confirms local chat is ready.

Connect an OpenAI-compatible app by entering the URL below as its server address. Use it on the same PC running PolyStrata. An Anthropic-compatible API¹³ is also available; follow the installation document's connection settings for that protocol.

OpenAI-compatible API base URL

http://127.0.0.1:8080/v1

If the first run fails, check model loading and GPU selection

If the browser cannot connect, first check whether the server window is still loading the model. The first load can take several minutes. If an error appears, keep the window open and read the last error line. If a model download stopped midway, rerun the setup file for your operating system to resume it.

For a CUDA compilation failure, check the last setup error for missing tools or an unsupported GPU. On a Linux NVIDIA PC with the CUDA 12 Toolkit installed, you can rebuild with the CUDA 12 option above.

If both GPUs do not appear, compare the card numbers in the GPU list with the selection saved during setup. Two-GPU operation was tested on Linux with NVIDIA cards.

Terminology notes

System RAM — System memory that temporarily holds data while programs run.
Back to the text
Prompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the text
MoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the text
Token — A unit into which a model divides input or output for processing.
Back to the text
GPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the text
CPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.
Back to the text
VRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the text
SSD — A data storage device that uses flash memory and retains data without power.
Back to the text
CUDA — A software platform for general-purpose computing on NVIDIA GPUs.
Back to the text
Backup — Saving settings and data to a file so the same setup can be returned to after a hardware failure.
Back to the text
Quantization — A way to represent model weights or values with fewer bits, reducing storage and memory use. The coarser representation can also affect accuracy.
Back to the text
Context window — The token span of input and generated content a model can handle in one request. The supported limit and memory use depend on the model and runtime settings.
Back to the text
API — A defined interface that lets other code call a program’s functions.
Back to the text

Frequently asked questions

What should you check first in this article?: With an 8GB graphics card and 32GB of system RAM, you can try running Qwen3.6 or Gemma 4. Choose a model size that fits your RAM, install it, and send a prompt in the browser.
How much memory does it need?: MoE models such as Qwen3.6 select a subset of expert weights for each token. PolyStrata caches frequently used experts in GPU memory and computes the rest on the CPU from system RAM. This lets a model file larger than 8GB run across GPU and system memory.