Executable programs and extensions
Last updated
Run Strata on an RTX 5090 32 GB: RAM choices and published results
The RTX 5090’s 32 GB VRAM keeps more experts on the GPU, while long prompts still need system-RAM headroom.
Strata is a local runtime1 that takes text and optional images and returns answers from Qwen3.8-Flash-Next. An RTX 5090 with 32 GB VRAM2 can keep more experts on the GPU3, while system RAM4 holds the remaining experts and runtime state. Strata recommends IQ2_XS at 64 GB RAM and offers IQ3_S or UD-IQ4_XS at 96 GB or more. A public stock Strata RTX 5090 report used version 0.1.29, IQ2_XS, Linux, and 64 GB RAM to measure three prompt5 lengths on a synthetic code-explanation task.
System RAM still matters with 32 GB VRAM
Strata runs on Windows and Linux, taking text and optional image input and returning Qwen3.8-Flash-Next answers to local chat or connected apps. The RTX 5090’s 32 GB VRAM can cache more experts and reduce CPU6 work during generation. System RAM still holds the full expert set, operating system, apps, and long-context state.
Long documents, code repositories, and other open apps need additional system RAM and KV cache7. Choose your normal apps and context length8 first, then leave enough RAM headroom when selecting IQ2_XS or IQ3_S.
General starting requirements are 12 GB VRAM, 32 GB RAM, and about 80 GB free disk space. Downloads and RAM needs vary by model; UD-IQ4_XS is about a 94 GB download and needs more storage. Check the Strata model guide for current recommendations.
Starting models at 64 and 96 GB RAM
At 64 GB RAM, IQ2_XS is the recommended starting model; Q2_0 is a faster option. At 96 GB or more, consider IQ3_S. It retains every expert and uses a larger model size, giving more room for answers and long-context work, though a 64 GB PC may need other apps closed. UD-IQ4_XS is another option at 96 GB, but its download is about 94 GB and it needs substantial RAM and storage.
With 32 GB system RAM and a GPU with at least 24 GB VRAM, Strata documents low-RAM configurations for Q2_0 and IQ2_XS. The RTX 5090’s 32 GB VRAM is within that range. Setup offers a mode that keeps the full model in RAM; leave memory for other apps and long prompts or image input.
Setup also downloads the MTP9 draft files. They propose tokens10 that the main model verifies to speed up generation, and the draft layer and head use some VRAM. Long-context KV cache also takes memory, so check VRAM, RAM, and first-request latency in Monitor while running your workload.
A 32K input took about six seconds to the first token
A published stock Strata community report used IQ2_XS on an RTX 5090 32 GB, Core Ultra 9 285K, 64 GB RAM, and Ubuntu Linux. It used engine 0.1.29, a 131,072-token context, INT8 KV, and MTP. The report gives medians from three runs each at 4K, 32K, and 128K prompts. The workload was synthetic code explanations with 256 generated tokens each.
Reported output speeds were 179.4 tok/s with a 4K prompt, 175.7 at 32K, and 165.0 at 128K. Prompt throughput11 at those lengths was 4,269.8, 5,543.2, and 5,778.7 tok/s. Although the 128K prompt had higher tokens-per-second throughput, it contained far more tokens and took about 22 seconds to reach the first response.
Scroll sideways to compare all columns.
| Prompt | Prompt throughput | Answer generation | Time to first token |
|---|---|---|---|
| 4K | 4,269.8 tok/s | 179.4 tok/s | 0.974 s |
| 32K | 5,543.2 tok/s | 175.7 tok/s | 5.950 s |
| 128K | 5,778.7 tok/s | 165.0 tok/s | 22.262 s |
Conditions: 2026-09-30 · RTX 5090 32 GB · Core Ultra 9 285K · 64 GB RAM · Ubuntu 24.04.4 · driver 595.84 · Strata 0.1.29 · IQ2_XS · INT8 KV · MTP · synthetic code explanation · fresh prompts · 256-token output cap · median of three runs per length.
Keep NVFP4 fork results separate from stock Strata
The RTX 5090 fork sergqwer/strata-nvfp4 uses an NVFP412 model pack and a modified engine. Prepare its engine and model files together, and retain results from standard Strata as a baseline when trying the fork.
A Windows report using sergqwer/strata-nvfp4 used an NVFP4 PLE13 file. Diverse prompts read the PLE table from disk and processed about 1,240 tokens/s; a synthetic prompt with repeated text reached about 6,500–7,300 tok/s because its rows stayed cached. --ple-io mmap reached about 6,037 tok/s in a warm run14, while the first cold pass had different behavior. Prompt type and cache state differed across these measurements.
If a real long prompt feels slow, check first-request context length, prompt throughput, RAM use, and SSD15 reads separately from output tok/s. Follow-up turns may reuse prompt state, so their wait differs from the first long request. Monitor can help show whether the GPU is busy while the CPU is idle or the CPU is computing experts.
Terminology notes
Runtime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textVRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textSystem RAM — System memory that temporarily holds data while programs run.
Back to the textPrompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the textCPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.
Back to the textKV cache — Memory that stores attention keys and values from earlier tokens for reuse during later token generation. Its size depends on context length and batch size.
Back to the textContext window — The token span of input and generated content a model can handle in one request. The supported limit and memory use depend on the model and runtime settings.
Back to the textMTP — A training objective or model component for predicting multiple future tokens at once. In speculative decoding, those predictions can be proposed as candidate tokens.
Back to the textToken — A unit into which a model divides input or output for processing.
Back to the textThroughput — The amount of work processed or generated over time. Comparisons need the unit, such as tokens per second or requests per second.
Back to the textNVFP4 — A 4-bit floating-point data format defined by NVIDIA. Support depends on GPU generation, model, and software implementation.
Back to the textPLE embedding table — Short for Per-Layer Embedding, a structure that adds embedding information at particular model layers. The Qwen Flash-Next family uses n-gram tables; supported runtimes can offload these tables to RAM or SSD.
Back to the textWarm run — A measurement made after model loading and initialization. It may exclude the loading wait from the first run.
Back to the textSSD — A data storage device that uses flash memory and retains data without power.
Back to the text
Frequently asked questions
- What should you check first in this article?
- Strata is a local runtime that takes text and optional images and returns answers from Qwen3.8-Flash-Next. An RTX 5090 with 32 GB VRAM can keep more experts on the GPU, while system RAM holds the remaining experts and runtime state. Strata recommends IQ2_XS at 64 GB RAM and offers IQ3_S or UD-IQ4_XS at 96 GB or more. A public stock Strata RTX 5090 report used version 0.1.29, IQ2_XS, Linux, and 64 GB RAM to measure three prompt lengths on a synthetic code-explanation task.
- How much memory does it need?
- Strata runs on Windows and Linux, taking text and optional image input and returning Qwen3.8-Flash-Next answers to local chat or connected apps. The RTX 5090’s 32 GB VRAM can cache more experts and reduce CPU work during generation. System RAM still holds the full expert set, operating system, apps, and long-context state.