Executable programs and extensions
Last updated
Run Strata on an RTX 3090 24 GB: VRAM, RAM, and long context
The RTX 3090’s 24 GB VRAM keeps more experts on the GPU, while RAM, CPU, and PCIe affect practical speed and headroom.
Strata runs Qwen3.8-Flash-Next locally from text and optional image prompts1. The RTX 3090’s 24 GB VRAM2 can cache more experts than a 12 GB card. Strata’s published 100–140 tok/s range assumes a CPU3 and 64 GB RAM4, with separate estimates by prompt length. A 32 GB system with an RTX 3090 can use low-RAM mode for Q2_0 or IQ2_XS. With 64 GB, start at IQ2_XS; with 96 GB, consider IQ3_S or UD-IQ4_XS.
What 24 GB of VRAM does for model inference
Strata is a Windows and Linux runtime5 that takes a prompt or optional image and generates an answer with Qwen3.8-Flash-Next. The RTX 3090 has 24 GB VRAM, allowing it to cache more experts than the 12 GB RTX 5070. RAM holds the remaining experts and runtime data; the CPU computes experts outside the GPU6 cache; the SSD7 stores PLE8 (per-layer embeddings) lookup data.
Generation speed depends on GPU memory bandwidth9, CPU expert throughput10, PCIe link width and generation, and available RAM. When moving a 3090 to another PC, check its CPU and motherboard link as well.
General requirements are at least 12 GB VRAM, 32 GB system RAM, and about 80 GB free disk space. When choosing a model for the RTX 3090, check whether RAM can hold the operating system, apps, and model experts alongside GPU capacity. Official requirements.
Choose a model for 32, 64, or 96 GB RAM
With an RTX 3090 and 32 GB system RAM, Strata documents low-RAM configurations for Q2_0, IQ2_XS, and Coder. This mode keeps experts that do not fit on the GPU resident in RAM, avoiding SSD reads during replies. If RAM is too tight for the runtime and other apps, the mmap path uses the OS file cache and may read more from SSD. Check RAM and disk activity in Monitor; close apps or choose a smaller model to free memory.
At 64 GB RAM, IQ2_XS is the recommended starting point; test IQ3_XXS or IQ3_S with your actual apps and context. IQ3_S may require closing other apps. At 96 GB or more, IQ3_S has more room, and the larger UD-IQ4_XS is also an installer option. On a 64 GB PC, UD-IQ4_XS reads some experts from SSD, so NVMe11 helps, and it requires more free space than the roughly 80 GB general starting allowance.
Coder is easier to fit on a 32 GB PC, but it removes half the experts and has reported weaker results than the original outside coding and in languages other than English. For Korean and general multilingual use, first compare full-expert models such as Q2_0 or IQ2_XS.
The 3090 speed estimate depends on CPU and PCIe
Strata’s RTX 3090 Q2_0 table separates prompt processing and answer generation by prompt length. Estimated output is about 140 tok/s at 4K, 130 at 32K, 100 at 128K, and 85 at 262K. With 128K or 262K context, include the time spent reading the initial prompt.
| Prompt length | Estimated prompt throughput | Estimated output |
|---|---|---|
| 4K | ~491 tok/s | ~140 tok/s |
| 32K | ~521 tok/s | ~130 tok/s |
| 128K | ~495 tok/s | ~100 tok/s |
| 262K | ~453 tok/s | ~85 tok/s |
The estimate carries roughly ±20% uncertainty, and its prompt figures use an older prefill12 path. The PCIe generation and link width affect how quickly experts reach the GPU. If changing only the GPU does not produce the expected speed, check CPU expert computation and the PCIe link reported by your system or Monitor.
Conditions: Strata’s published RTX 3090 figures assume a CPU and 64 GB RAM, with roughly ±20% uncertainty; they are estimates rather than measurements. Prompt figures use an older path. Official RTX 3090 estimates.
Check KV cache options for long context
In long coding conversations, the KV cache13 grows and uses RAM or VRAM. At longer context lengths, Strata can keep KV in RAM and only part resident in VRAM, leaving more room for the expert cache. This changes where the cache lives, not the answer values, and uses system RAM.
K8V4 is an optional RTX 3090 option. In a reported run of Coder at 198K context, it used 23% less KV memory than 8-bit KV and raised output from 85 to 99 tok/s. Needle-retrieval results were unchanged in that test, while prompt processing was 2–5% slower. K8V4 is optional; check long-document accuracy and your own workload before choosing it.
If long context exhausts RAM or keeps reading from disk, reduce context and close other apps. Compare the answer, memory use, and completion time with default KV and K8V4 on the same input to choose for your work.
Terminology notes
Prompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the textVRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the textCPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.
Back to the textSystem RAM — System memory that temporarily holds data while programs run.
Back to the textRuntime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textSSD — A data storage device that uses flash memory and retains data without power.
Back to the textPLE embedding table — Short for Per-Layer Embedding, a structure that adds embedding information at particular model layers. The Qwen Flash-Next family uses n-gram tables; supported runtimes can offload these tables to RAM or SSD.
Back to the textMemory bandwidth — The amount of data transferable between memory and a processor per unit time, usually expressed in bytes per second (B/s).
Back to the textThroughput — The amount of work processed or generated over time. Comparisons need the unit, such as tokens per second or requests per second.
Back to the textNVMe — A storage protocol used by SSDs communicating over PCI Express.
Back to the textPrefill — The stage where an LLM reads the input prompt and computes representations for its tokens. Longer prompts contain more tokens to process.
Back to the textKV cache — Memory that stores attention keys and values from earlier tokens for reuse during later token generation. Its size depends on context length and batch size.
Back to the text
Frequently asked questions
- What should you check first in this article?
- Strata runs Qwen3.8-Flash-Next locally from text and optional image prompts. The RTX 3090’s 24 GB VRAM can cache more experts than a 12 GB card. Strata’s published 100–140 tok/s range assumes a CPU and 64 GB RAM, with separate estimates by prompt length. A 32 GB system with an RTX 3090 can use low-RAM mode for Q2_0 or IQ2_XS. With 64 GB, start at IQ2_XS; with 96 GB, consider IQ3_S or UD-IQ4_XS.
- How much memory does it need?
- Strata is a Windows and Linux runtime that takes a prompt or optional image and generates an answer with Qwen3.8-Flash-Next. The RTX 3090 has 24 GB VRAM, allowing it to cache more experts than the 12 GB RTX 5070. RAM holds the remaining experts and runtime data; the CPU computes experts outside the GPU cache; the SSD stores PLE (per-layer embeddings) lookup data.