new model · 2026.09.04
Last updated
Gemma 4: choosing 12B or 26B-A4B for your hardware
Choose Gemma 4 for the way it handles images and documents together
Gemma 4 is a family of open-weight models that handles text and image or audio inputs. You can provide a machine warning-light photo and its manual and ask for inspection steps, but the chosen app and model file must support passing the image through.
The family includes E2B, E4B, 12B, 26B-A4B, and 31B sizes, with context up to 256K. This guide uses the 12B dense and 26B-A4B MoE1 models for a photo-and-manual task to show how to check input delivery and memory conditions.
Bring the photo and manual together in one question
If a machine shows a warning light, provide its photo and the relevant manual section, then ask what indicator is visible and what inspection order the manual specifies. Check that the response distinguishes what appears in the image from the manual's procedure.
First send only the manual text and check whether the model finds the needed procedure. Then add the photo and test whether the app passes image input to the model. Family-level image and audio support does not ensure that every size, converted file, and runtime2 combination handles those inputs. If the response behaves as though it received text only, inspect the input path.
Dense and MoE models differ in weights and computation
The 12B dense model uses its full model path to compute a token3. The roughly 26B MoE activates about 4B per token. The active count describes computation for that token, not the full file size; later tokens may need other experts, so the runtime needs access to the full weights.
The roughly 24 GB official 12B source-weight figure is not the size of a desktop Q4 conversion. The storage precision and file format differ. Check the actual file size and allow additional runtime space for the KV cache4, which depends on context, and other runtime needs.
A rough calculation assuming only raw 4-bit weights is parameter count × 4 ÷ 8. In decimal GB, that gives 6 GB for 12B and 13 GB for 26B, but these are not actual download sizes or runtime memory. Block metadata and scales, components stored at other precisions, vision input, and runtime needs are not included.
On 16 GB, start by testing 12B Q4 with your usual apps
A 16 GB-class system is a starting point for examining 12B Q4. Whether it loads depends on the file, runtime, and other programs sharing memory. Begin with a short manual excerpt and one request, then try the same question with your usual browser and document apps open. A Mac's unified memory5 is shared by CPU6, GPU7, operating system, and apps.
The same 16 GB figure means different things for discrete-GPU VRAM8 and Mac unified memory. A PC has separate system RAM9, but display output or other GPU work may use some VRAM; on a Mac, CPU and GPU share unified memory. Before changing models, use the runtime's display and logs to see which memory is constrained.
At 24 GB or more, compare 26B-A4B using the same photo
At 24 GB or more, 26B-A4B Q4 is a starting point for comparison. Give both candidates the same photo and manual section, and check whether each correctly identifies the visible indicator and the document's inspection steps. Do not judge by a longer response or larger model name alone; define the required answer criteria first.
Record the model file and app, manual excerpt, whether the image reached the model, and which evidence each answer missed. If one candidate never received the image, the result does not compare model sizes. A larger candidate is worth examining when the input path is consistent, 12B repeatedly misses an important condition, and the runtime has memory headroom.
The model is an assistant for finding information in photos and manuals. It does not replace diagnosis or repair decisions, so verify safety-related steps against the manufacturer's manual and a responsible person's judgment.
Terminology notes
MoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the textRuntime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textToken — A unit into which a model divides input or output for processing.
Back to the textKV cache — Memory that stores attention keys and values from earlier tokens for reuse during later token generation. Its size depends on context length and batch size.
Back to the textUnified memory — An architecture where the CPU and GPU share one physical memory pool.
Back to the textCPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textVRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the textSystem RAM — System memory that temporarily holds data while programs run.
Back to the text
Frequently asked questions
- What should you check first in this article?
- Gemma 4 is a family of open-weight models that handles text and image or audio inputs. You can provide a machine warning-light photo and its manual and ask for inspection steps, but the chosen app and model file must support passing the image through.
- How much memory does it need?
- A rough calculation assuming only raw 4-bit weights is parameter count × 4 ÷ 8. In decimal GB, that gives 6 GB for 12B and 13 GB for 26B, but these are not actual download sizes or runtime memory. Block metadata and scales, components stored at other precisions, vision input, and runtime needs are not included.