Qwen-Image 2.1 GGUF: Unsloth setup guide

The Unsloth GGUF³ repository contains the denoiser that builds an image from noise. A separate Qwen3-VL-8B encoder reads the prompt; the VAE converts the compressed representation into pixels. A valid denoiser with a mismatched encoder or VAE can still fail or produce broken output. Compatibility comes before file size.

For 2.1, use qwen_image_2.1_vae_bf16.safetensors, not an older Qwen-Image or Wan 2.2 VAE. The UD-Q4_K_XL encoder is also separate from the Q4_K_M denoiser. Recording the model name and each file's role helps avoid duplicate downloads when changing runtimes.

Q4_K_M reduces the denoiser to about 4.2GB. Q5_K_M is 5.39GB, Q6_K 6.27GB and Q8_0 7.64GB. Smaller weights save storage and memory but do not eliminate dequantization costs. Lower precision is not automatically faster or visually identical.

Unsloth retains higher precision in sensitive parts, but that does not guarantee pixel-identical output. Keep familiar prompts and seeds fixed and inspect lettering, contours and patterns. One published Mac setup ran Q8 faster than Q4 at 1K; ample RAM⁴ is a reason to test both, not select by file size alone.

These commands require the Hugging Face hf tool. They download only the Q4 denoiser, Q4 encoder and VAE under models. Complete any requested authentication or license acceptance on the repository page first. Existing files can be reused by changing the paths in the run command.

The three files occupy roughly 10GB on disk. Download size is not runtime memory: 10GB free storage does not establish that inference⁵ will fit. Leave extra storage for additional formats, and remember that macOS and other apps share unified memory⁶ with the model.

sd-cli⁷ is the stable-diffusion.cpp executable. Use a Qwen-Image 2.1-capable version built for CUDA⁸ or Metal; the example assumes sd-cli is on PATH. On Windows, use the path to sd-cli.exe. Run from the directory containing models. This creates a local image, not a publicly exposed server.

The starting recipe follows Unsloth's sd.cpp example: 20 steps, CFG 6.0, Euler, 1024 square. It is not a universal optimum. Align settings before comparing timing against 40-step Diffusers or LightX2V runs. Verify one successful image first, then change steps or resolution one at a time.

Avoid starting with 2K or concurrent images. Try one image at 1024 or below with Q4 and offloading⁹, pausing other GPU¹⁰ tasks. If memory still runs out, check where the encoder is loaded before further compressing the denoiser. A large encoder on the same GPU can consume the expected savings.

Offloading makes more configurations runnable; it is not free acceleration. CPU¹¹ RAM and transfer speed affect the wait. On Mac, heavy swapping can feel slow despite ample SSD¹² space. A missing time estimate for a low-memory configuration is not a claim that it cannot run, but a refusal to promise a stable swap-dependent duration.

If text-to-image works but reference images fail, check the vision projector. sd.cpp documents a separate mmproj supplied through --llm_vision for editing with GGUF encoders. Match it to the encoder model; filenames vary by distributor. Redownloading the denoiser alone will not solve this.

Pass references with -r, repeating it for multiple images. First verify a one-image background edit. Preserve alpha in PNG or WebP. If your ComfyUI node or desktop app lacks a feature, distinguish model capability from application support and use a compatible workflow or runtime.

The first run includes loading weights. Where possible, record at least three subsequent generations and take the median. Full sd-cli process time, which may reload weights each invocation, differs from a warmed server request. Recording the generation phase separately makes updates easier to compare.

Save filenames, runtime version, encoder, VAE, resolution, steps, seed and offload settings. Replacing hardware before separating configuration problems from GPU limits can reproduce the same issue. This is an installation guide, not a claim of our own measurements. The original research license still applies; check separate commercial permission when relevant.