What should you check first in this article?

Strata is a local runtime for Windows and Linux that runs Qwen3.8-Flash-Next and returns answers to a browser or compatible app. The GPU caches frequently used experts, RAM and the CPU handle experts outside the GPU, and the SSD supplies the selected model and PLE data. This guide explains how to choose a setup for your RAM, GPU, and operating system, then send a first question locally.

Executable programs and extensions

A 125B model on a 12GB GPU? How Strata runs it

Even when GPU memory cannot hold the entire 125B model, Strata distributes the work across PC components.

Strata is a local runtime¹ for Windows and Linux that runs Qwen3.8-Flash-Next and returns answers to a browser or compatible app. The GPU² caches frequently used experts, RAM³ and the CPU⁴ handle experts outside the GPU, and the SSD⁵ supplies the selected model and PLE⁶ data. This guide explains how to choose a setup for your RAM, GPU, and operating system, then send a first question locally.

Strata is a local runtime for Windows and Linux PCs. It runs Qwen3.8-Flash-Next and returns answers to text prompts⁷ or, when enabled, image inputs through a browser or connected app. Its default server address, http://127.0.0.1:8080, is accessible from the PC itself.

The model's full weights can exceed the memory of one GPU, so Strata assigns different roles to the GPU, system RAM, CPU, and storage. Understanding those roles helps you judge which setup is practical on your PC and where a slow response is coming from.

Contents

Requirements and key details

Strata runs the Qwen3.8-Flash-Next 125B MoE model on a PC by using GPU, RAM, CPU, and storage in different roles.
For RAM capacity, compare Coder at 32 GB, IQ2_XS or Q2_0 at 48 GB, and start with IQ2_XS at 64 GB.
Strata's code and model weights may come from different distributors and have different licenses, so check each one separately.

Only part of the 125B model is used for each token

Qwen3.8-Flash-Next has a roughly 125B-parameter MoE⁸ language-model core. Including PLE embeddings and MTP⁹ components brings the full model to about 180B. The core divides its weights into expert groups and selects a subset for each token¹⁰, so only some experts perform each token's computation.

Strata caches experts in GPU memory so frequently used parts can be processed quickly, while the CPU and system RAM handle experts that are not loaded on the GPU. When you send a question, the local server passes it to the model and returns the answer to the browser's Chat page or a compatible app. You can use text alone or enable image input; support depends on the GPU and setup.

As a starting point, plan for at least 12 GB of VRAM¹¹ on a supported GPU, 32 GB of RAM, and about 80 GB of free disk space. The project documents Windows and Linux, and GPU support may change between versions. Check the current requirements and setup steps in the Strata repository and installation guide.

A black tower PC, a blank monitor, and RAM modules on a warm wood desk indoors — Strata organizes a PC to run the 125B MoE model locally.

The GPU, RAM, CPU, and SSD each have a different role

In a standard setup, expert weights are kept in system RAM and frequently used experts are cached in GPU memory. The CPU helps calculate experts that are not on the GPU. A larger GPU cache lets the GPU handle more expert work, while the full model weights and working memory still need to fit across the system.

Low-RAM mode determines whether experts that do not fit in the GPU are supplied from RAM or from a file. If the remaining expert weights fit in available RAM, resident mode pins them in RAM at startup. It therefore does not read experts from the SSD during normal responses.

If RAM is insufficient, mmap mode maps the file and reads the portions needed by the GPU from the file cache. Generation can slow down when VRAM is small and the GPU frequently needs experts that are not resident there. The installer selects a mode based on available RAM, and START-HERE.bat --setup --low-ram on|off changes the low-RAM setting.

PLE (per-layer embeddings) is a table of embeddings used by the model's layers. In Strata, this data can live on an SSD and be read as needed, leaving more GPU and RAM capacity for weights. Storage performance and the operating system's file cache affect these reads, so an NVMe¹² SSD with enough free space can help. Depending on the setup, PLE file reads take place alongside GPU and CPU computation.

Watch GPU, CPU, and RAM use in Monitor while a response is running. If RAM is nearly full and disk reads continue, close memory-heavy apps or choose a smaller model. Less GPU memory available for the expert cache can also shift more work to the CPU. The technical overview describes Strata's memory tiers and low-RAM modes.

For Intel Arc or Radeon on Linux, check XeStrata's setup requirements. Match the GPU and distribution before following the relevant setup path.

A graphics card, RAM modules, and an NVMe SSD on a warm wood desk — Consider the GPU cache, RAM and CPU work, and SSD-hosted layer embeddings separately.

Choose a model variant for your RAM and language needs

In the model selector, choose a quantization¹³ size that fits your RAM and workload. Q2_0, IQ2_XS, and IQ3_S retain all experts in quantized form. Coder is tuned for coding and keeps 256 of the 512 experts. A smaller model needs less storage and memory, but Coder may produce different results from the full model on general chat and some languages.

Strata's current model guide suggests Coder for 32 GB RAM, IQ2_XS or the faster Q2_0 for 48 GB, and IQ2_XS as a starting point for 64 GB. A 64 GB PC can run larger variants, though IQ3_S with a long context leaves less room for the OS and apps. With 96 GB or more, consider IQ3_S or Unsloth UD-IQ4_XS; UD-Q4_K_XL is an experimental configuration. Include the context's KV cache¹⁴ and running apps in your RAM budget, not just the model file.

With 32 GB of RAM and a 24 GB GPU, Q2_0 or IQ2_XS can be run in low-RAM mode. With 32 GB of RAM and a 12–16 GB GPU, Coder is a practical first option. If a setup frequently reads experts that do not fit on the GPU from the SSD, generation can remain slow even with NVMe storage. Consider the languages you use and how long you can wait for a response.

If you regularly work with Korean or Japanese text and code, compare Q2_0 or IQ2_XS with Coder using the same prompt instead of choosing Coder alone. Swift 1.5 is tuned to reduce the thinking time before a response; its token generation rate is similar to Q2_0, but it may answer sooner. Results and licenses vary by model variant, so check the download page and model selection guide.

In 0.1.43.1, enable adaptive overlap only where it helps

From 0.1.43.1, STRATA_ADAPT_OVERLAP is off by default on every machine after the 0.1.43 default slowed decode¹⁵ on some Linux NVIDIA PCs. You can test STRATA_ADAPT_OVERLAP=1: check its state in the startup log and compare the same model and prompt with it off and on. The 0.1.43 results below retain their original measurement conditions.

0.1.43.1 hotfix and conditions

Read the 0.1.43 gains by GPU and workload

Strata 0.1.43 was released on October 11, 2026. The table below summarizes changes to prompt processing and answer generation across NVIDIA, AMD, and Intel setups.

GPU-specific results reported for Strata 0.1.43
GPU and setup	Model and conditions	Change in 0.1.43
Radeon AI PRO R9700 32 GB · 16 GB-class cache	IQ3_XXS	Decode +21–38%; prompts: +7.5% at 4K, +15.6% at 32K
Radeon AI PRO R9700 · 32 GB cache	Q2_0 / IQ3_XXS	Decode +2–10%
Radeon AI PRO R9700 · 32 GB cache	IQ3_S	Decode +6–14%; prompts: +17% at 32K, +27% at 64K
RTX 5070 · Windows defaults	Q2_0 / IQ3_XXS	No default decode change
Arc Pro B70	IQ3_S · 128K context	Decode +14–21%; 4K prompt: 8.5→3.3 s; 16K: 27.4→13.4 s

The R9700 has 32 GB of physical VRAM. The “16 GB-class cache” row limits its expert cache to the size used on a 16 GB card. Results are medians from five interleaved old/new pairs; Arc Pro B70 prompt tests use three pairs.

On Windows, the RTX 5070 has the same default decode behavior as 0.1.42. Two NVIDIA changes are not enabled by default there. On Linux, adaptive-tier copies overlap the next decode window, and the PCIe share is measured and applied automatically. On Windows, you can opt into the corresponding settings with STRATA_ADAPT_OVERLAP=1 and STRATA_PCIE_AUTO_CANON=1. Adaptive overlap was 2.6% slower for Q2_0 on a Windows RTX 5070, so it remains off by default.

R9700 results vary with the cache limit and quantization. Reading the limited-cache and actual 32 GB-cache rows separately shows how the number of experts held on the GPU relates to the result. The Arc Pro B70 also improved prompt processing. The table keeps the exact figures with their model and context conditions.

The full results and test conditions are in the v0.1.43 release notes and detailed measurements.

Check Monitor and local chat after setup

For a new installation, prepare the engine and model with START-HERE.bat on Windows or ./setup.sh from the repository folder on Linux.

To update a branch-based Git installation without a fixed version tag, use UPDATE.bat or ./update.sh in that same Strata folder.

For a fixed-tag or ZIP installation, move to a new release ZIP. On first setup, choose a model and context that fit your RAM and VRAM; if you do not need image input, turn off vision to leave more GPU memory for the expert cache. See the Windows and Linux installation guide for the file selection and steps.

After setup, open http://127.0.0.1:8080 in a browser and send a short question in Chat. When the question and answer appear, the local server and model have processed the request. Compatible apps use http://127.0.0.1:8080/v1 as the server URL. If another device will connect, follow the address and authentication guidance in the API documentation; exposing a port on the network changes who can reach the service from its local-only default.

If a first-install download is interrupted, rerun START-HERE.bat on Windows or ./setup.sh on Linux. Use UPDATE.bat or ./update.sh to update a branch-based Git installation; move fixed-tag or ZIP installations to a new release ZIP. If the server does not open, check whether another program is using port 8080. If model loading or a response stops, check free RAM and VRAM, the selected model, and the context length¹⁶. See the official troubleshooting guide for specific symptoms.

Advanced Linux settings for multiple Radeon GPUs

Linux --serve with several Radeons using layer splitting is an advanced server setup. Strata 0.1.43 enables --pipeline-windows 2 by default on three or more gfx12 Radeons. With two cards, measured code generation was 3% slower, so it remains off by default and can be enabled manually.

In a separate test with four cards, each R9700 was set to 225 W and IQ3_S was used for a short story, code, and a 4K document. The pipeline¹⁷ changes the size of expert-row batches sent to the CPU in each window. Because the order of calculations changes, summation and rounding can differ; some answers may vary slightly from serial execution. See the AMD HIP guide and 0.1.43 detailed measurements for setup and performance conditions.

AMD HIP and Radeon measurements

Two graphics cards and other components installed in an open PC case — On a PC with two GPUs, check the GPU list and layer-splitting settings used by the runtime.

Check the separate licenses for code and model files

The Strata repository code is distributed under the MIT License. Model weights follow the terms of their own distributors, so check the model card license before downloading the original Qwen model, Coder, Swift 1.5, or an Unsloth variant. The code license and the model's use or redistribution terms are separate checks.

License information is available in the project documentation, the repository license, and each model distribution page.

For installation steps, continue with the Windows and Linux installation guide; for quantization sizes and RAM choices, see the model selection guide.

Terminology notes

Runtime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the text
GPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the text
System RAM — System memory that temporarily holds data while programs run.
Back to the text
CPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.
Back to the text
SSD — A data storage device that uses flash memory and retains data without power.
Back to the text
PLE embedding table — Short for Per-Layer Embedding, a structure that adds embedding information at particular model layers. The Qwen Flash-Next family uses n-gram tables; supported runtimes can offload these tables to RAM or SSD.
Back to the text
Prompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the text
MoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the text
MTP — A training objective or model component for predicting multiple future tokens at once. In speculative decoding, those predictions can be proposed as candidate tokens.
Back to the text
Token — A unit into which a model divides input or output for processing.
Back to the text
VRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the text
NVMe — A storage protocol used by SSDs communicating over PCI Express.
Back to the text
Quantization — A way to represent model weights or values with fewer bits, reducing storage and memory use. The coarser representation can also affect accuracy.
Back to the text
KV cache — Memory that stores attention keys and values from earlier tokens for reuse during later token generation. Its size depends on context length and batch size.
Back to the text
Token decoding — The stage that generates successive output tokens using the input and the response produced so far.
Back to the text
Context window — The token span of input and generated content a model can handle in one request. The supported limit and memory use depend on the model and runtime settings.
Back to the text
Pipeline — A sequence of processing stages from input to output. Different models or tools may be used at each stage.
Back to the text

Frequently asked questions

What should you check first in this article?: Strata is a local runtime for Windows and Linux that runs Qwen3.8-Flash-Next and returns answers to a browser or compatible app. The GPU caches frequently used experts, RAM and the CPU handle experts outside the GPU, and the SSD supplies the selected model and PLE data. This guide explains how to choose a setup for your RAM, GPU, and operating system, then send a first question locally.
How much memory does it need?: As a starting point, plan for at least 12 GB of VRAM on a supported GPU, 32 GB of RAM, and about 80 GB of free disk space. The project documents Windows and Linux, and GPU support may change between versions. Check the current requirements and setup steps in the [Strata repository](https://github.com/Niko1221/Strata) and [installation guide](https://github.com/Niko1221/Strata/blob/main/docs/INSTALL.md).
Where can you check the conditions and sources?: If you regularly work with Korean or Japanese text and code, compare Q2_0 or IQ2_XS with Coder using the same prompt instead of choosing Coder alone. Swift 1.5 is tuned to reduce the thinking time before a response; its token generation rate is similar to Q2_0, but it may answer sooner. Results and licenses vary by model variant, so check the download page and [model selection guide](https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#pick-by-ram).