Executable programs and extensions
Last updated
A 35B model on 4 GB VRAM? XeStrata Ornith and llm-jp requirements
4 GB VRAM is the minimum path for specific beta quantizations. You still need enough system RAM and disk space for the model weights.
XeStrata's 4 GB figure applies only to specified beta files such as Ornith-1.5 35B-A3B Q4_K_M and llm-jp-4.1 32B-A3B Q4_K_M. Their downloads are about 21.7 GB and 21.5 GB, and each calls for about 30 GB of system RAM1. A 4 GB GPU2 does not make Qwen3.8 125B or every quantization3 runnable.
Even with 4 GB of VRAM4, XeStrata's beta MoE5 models can be tested using system RAM alongside the GPU. Compare file sizes and RAM requirements for Ornith-1.5 35B-A3B and Japanese-focused llm-jp-4.1 32B-A3B, choose a supported file, and send a first prompt6.
Start with files that meet the 4 GB condition
XeStrata's README lists model download size, RAM, and minimum VRAM separately for beta models. The entries marked 4 GB are Ornith-1.5 35B-A3B Q4_K_M and llm-jp-4.1 32B-A3B Q4_K_M. Their files are about 21.7 GB and 21.5 GB, and each calls for about 30 GB RAM.
The README also lists a smaller community reproduction. GSQ-RCO-3BIT is 13 GB on disk, 22 GB RAM, and 4 GB minimum VRAM, but has no MTP7 layer. Its source and MTP status differ from the standard official Q4_K_M file. Higher options include Q5_K_M at 25.3 GB/34 GB/6 GB, Q6_K at 29.2 GB/38 GB/6 GB, and Q8 at 37.8 GB/46 GB/6 GB.
Choose between Japanese use and exploring the model family
For Japanese documents or chat, start by considering llm-jp-4.1 32B-A3B Q4_K_M. The README lists a 21.5 GB file, about 30 GB RAM, and 4 GB minimum VRAM. BF168 is a 64.3 GB file requiring 73 GB RAM and 8 GB VRAM, so it does not fit the same 4 GB configuration.
Ornith-1.5 35B-A3B Q4_K_M has the same listed RAM and VRAM thresholds. Larger quantized files are not listed as 4 GB options, so begin with the matching Q4_K_M and a short prompt to check whether it runs before changing models. Both families are labeled beta and text-only in the README.
Check PC memory and installation path before downloading
XeStrata provides distribution packages for Ubuntu 26.04 and Fedora 44. The model file is about 22 GB, so leave room on the target drive and check that roughly 30 GB of system RAM is available. Other running applications reduce the RAM headroom below the installed total.
This is a beta path. The README says some low-memory-card checks were performed by limiting B70 VRAM and XMX paths, rather than running on a physical low-memory card. Do not read that as a reproduced 4 GB run on your exact GPU; verify setup and a short first generation on your own system.
After installation, use a short prompt to check memory headroom
Install the release package for your distribution, then run xestrata --setup. Check the detected GPU and select Q4_K_M for Ornith or llm-jp. Compare the displayed RAM recommendation and file size with the figures above before downloading.
xestrata --setupStart the first generation with a short Japanese question or a one- or two-sentence summary request. Open http://127.0.0.1:8095; a completed answer with no terminal memory error confirms the basic run. Save the prompt so you can compare if you change quantization or context length9.
Separate RAM errors from unsupported-path errors
If xestrata --setup cannot find a GPU library, compare the device name and gfx code from xestrata --check and your distribution with the README support table. AMD also needs rocBLAS for that gfx code in the distribution. If setup still will not open after installation, check the first terminal error and logs in ~/.local/state/xestrata.
If RAM is insufficient, close other applications and retry. If that does not help, check whether the README lists a smaller option on the same 4 GB path. If not, the practical choices are adding RAM or switching to a smaller model. Decide whether this setup is usable by whether a short Q4_K_M prompt completes.
Terminology notes
System RAM — System memory that temporarily holds data while programs run.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textQuantization — A way to represent model weights or values with fewer bits, reducing storage and memory use. The coarser representation can also affect accuracy.
Back to the textVRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the textMoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the textPrompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the textMTP — A training objective or model component for predicting multiple future tokens at once. In speculative decoding, those predictions can be proposed as candidate tokens.
Back to the textBF16 — A 16-bit floating-point format for model values. Its exponent has the same width as FP32, giving it a similarly wide value range.
Back to the textContext window — The token span of input and generated content a model can handle in one request. The supported limit and memory use depend on the model and runtime settings.
Back to the text
Frequently asked questions
- What should you check first in this article?
- XeStrata's 4 GB figure applies only to specified beta files such as Ornith-1.5 35B-A3B Q4_K_M and llm-jp-4.1 32B-A3B Q4_K_M. Their downloads are about 21.7 GB and 21.5 GB, and each calls for about 30 GB of system RAM. A 4 GB GPU does not make Qwen3.8 125B or every quantization runnable.
- How much memory does it need?
- XeStrata's README lists model download size, RAM, and minimum VRAM separately for beta models. The entries marked 4 GB are Ornith-1.5 35B-A3B Q4_K_M and llm-jp-4.1 32B-A3B Q4_K_M. Their files are about 21.7 GB and 21.5 GB, and each calls for about 30 GB RAM.