Executable programs and extensions
Last updated
Run Strata on an RTX 5070 12 GB: Model and RAM choices
An RTX 5070 can run Strata’s 125B MoE by sharing work across GPU, RAM, CPU, and SSD.
Strata is a local Windows and Linux runtime1 that takes text and optional image input and returns answers from Qwen3.8-Flash-Next. With 12 GB of VRAM2, an RTX 5070 caches the experts the GPU3 can reach quickly while system RAM4 holds the full set of experts and runtime data. In the published RTX 5070, Ryzen 5 7600, 64 GB RAM setup, Q2_0 generated 94 tok/s in short chat and 76 tok/s with 128K context. On a 32 GB RTX 5070 system, start by considering Coder; Q2_0 and IQ2_XS low-RAM mode is documented for a 24 GB GPU.
Strata turns prompts into answers on the RTX 5070
Strata is a local Windows and Linux runtime. It takes a text prompt5 or an optional image and sends Qwen3.8-Flash-Next’s answer to its browser chat or a connected app. The RTX 5070 has 12 GB of VRAM; Strata combines the GPU, system RAM, CPU6, and SSD7 to run this 125-billion-parameter MoE8 model.
VRAM caches experts for fast GPU access, while RAM holds the full set of model experts and runtime data. The CPU calculates experts outside the GPU cache, and the SSD stores the PLE9 (per-layer embeddings) lookup data. Choose a model by considering VRAM and system RAM together.
Strata’s general starting requirements are at least 12 GB VRAM, 32 GB system RAM, and about 80 GB of free disk space. An SSD shortens the first model load. The download is about 70 GB, so leave room for the rest of the files as well. Check the official Strata repository for current GPU and OS support.
- Strata project
- How GPU, RAM, CPU, and SSD work together
- official Strata repository
- MoE vs dense: why low active parameters still need memory
- A 125B model on a 12GB GPU? How Strata runs it
- Run Strata on an RTX 3090 24 GB: VRAM, RAM, and long context
- Which local LLM app or serving engine should you start with?
64 GB RAM gives you more model-size choices
The RTX 5070 meets Strata’s general VRAM minimum, but system RAM determines which model sizes fit. The installer recommends a size for your RAM. Coder, which keeps 256 of 512 experts, is the starting choice at 32 GB. At 48 GB, compare Q2_0 and IQ2_XS; at 64 GB, start with IQ2_XS.
With 64 GB RAM, IQ3_XXS and IQ3_S are also options. IQ3_S is larger, so you may need to close other apps. Windows, your browser, and an editor all use RAM; account for the apps you plan to keep open. Strata’s RAM selection guide says 96 GB or more gives IQ3_S more room.
Coder is a separate model tuned for coding. Keeping half the experts makes it easier to run with 32 GB RAM, but Strata’s project notes weaker results than the original outside English, including CJK text. For general questions and Korean writing, first compare sizes such as Q2_0 or IQ2_XS that retain all experts.
Short chats and 128K context have different speeds
This table shows how output speed and long-prompt processing changed across model sizes in Strata’s published RTX 5070 measurements. It separates short-chat output from output at 128K context; prompt throughput10 is for reading a 32K input.
Q2_0 generated about 94 tokens11 per second in short chat and about 76 at 128K context. IQ3_S reached about 53 and 46 tok/s in those conditions. For a first request containing a document or code history, the 32K prompt-throughput column helps compare how quickly the input is read before the reply begins.
Scroll sideways to compare all columns.
| Model size | Short-chat output | 128K-context output | 32K prompt processing | Suggested system RAM |
|---|---|---|---|---|
| Q2_0 | 94 tok/s | 76 tok/s | 2,650 tok/s | 48 GB |
| IQ2_XS | 79 tok/s | 63 tok/s | 2,090 tok/s | 48 GB |
| IQ3_S | 53 tok/s | 46 tok/s | 1,620 tok/s | 64 GB |
Conditions: RTX 5070 12 GB, Ryzen 5 7600, 64 GB DDR5-5200 RAM, Windows. Q2_0 used Strata 0.1.36; IQ2_XS and IQ3 rows used 0.1.26. Output measurements used 4K and 128K prompts; prompt throughput used 32K. The output cap was 256 tokens. Strata detailed measurements.
Install Strata and send a first prompt
On Windows, download and unpack Strata, then run START-HERE.bat. On Linux, run ./setup.sh from the project folder. Setup detects the GPU and asks you to choose a model size, context length12, and whether to enable image input. With an RTX 5070 and 64 GB RAM, begin with IQ2_XS, then compare IQ3_XXS or IQ3_S using your Korean tasks and available memory as guides.
The first setup downloads about 70 GB. The initial load can make the PC slow to respond for one to three minutes while the model is read into RAM. Keep the runtime window open until it finishes. Then visit http://127.0.0.1:8080 in a browser and send a short prompt. For an OpenAI-compatible app, use http://127.0.0.1:8080/v1 as the base URL.
If replies are very slow while the disk stays busy, RAM may be too tight and some experts may be read from the SSD. Close other apps or choose a smaller model to free memory. If a download stops, run the startup file again to resume. If port 8080 is already in use, check whether Strata is already running. See the Windows and Linux installation guide for setup steps and the Strata memory guide for capacity planning.
Terminology notes
Runtime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textVRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textSystem RAM — System memory that temporarily holds data while programs run.
Back to the textPrompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the textCPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.
Back to the textSSD — A data storage device that uses flash memory and retains data without power.
Back to the textMoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the textPLE embedding table — Short for Per-Layer Embedding, a structure that adds embedding information at particular model layers. The Qwen Flash-Next family uses n-gram tables; supported runtimes can offload these tables to RAM or SSD.
Back to the textThroughput — The amount of work processed or generated over time. Comparisons need the unit, such as tokens per second or requests per second.
Back to the textToken — A unit into which a model divides input or output for processing.
Back to the textContext window — The token span of input and generated content a model can handle in one request. The supported limit and memory use depend on the model and runtime settings.
Back to the text
Frequently asked questions
- What should you check first in this article?
- Strata is a local Windows and Linux runtime that takes text and optional image input and returns answers from Qwen3.8-Flash-Next. With 12 GB of VRAM, an RTX 5070 caches the experts the GPU can reach quickly while system RAM holds the full set of experts and runtime data. In the published RTX 5070, Ryzen 5 7600, 64 GB RAM setup, Q2_0 generated 94 tok/s in short chat and 76 tok/s with 128K context. On a 32 GB RTX 5070 system, start by considering Coder; Q2_0 and IQ2_XS low-RAM mode is documented for a 24 GB GPU.
- How much memory does it need?
- Strata is a local Windows and Linux runtime. It takes a text prompt or an optional image and sends Qwen3.8-Flash-Next’s answer to its browser chat or a connected app. The RTX 5070 has 12 GB of VRAM; Strata combines the GPU, system RAM, CPU, and SSD to run this 125-billion-parameter MoE model.