New model · 2026-09-04
Gemma 4: choosing 12B or 26B-A4B for your hardware
The Gemma 4 comes in several sizes, making it rather difficult to choose. You may be concerned that if you receive a small model, you'll miss out on the larger model's answers. However, it is also important to be able to keep it turned on and used on my device. Let's read the differences focusing on 12B and 26B-A4B.
Start with the task, not the model size
The conditions for a model that will organize short documents in a notebook and those that will continue to generate long code may be different. You also need to decide whether you need images or voice input in addition to text. Features introduced across the product family should not be assumed to be the same in all sizes and conversions.
At first, I send a few of the usual questions to a small group of candidates. The starting point is to see whether the answer is sufficient and whether it is easy to see what is wrong. If you leave something out just because it's small, you'll miss out on what you can do with the equipment you already have.
12B and 26B-A4B differ in more than size
12B is a dense model. 26B-A4B is an MoE that uses only part of its weights for each token. A larger total parameter count does not increase the work per token by the same proportion. But that does not mean you can load only the active weights into memory.
Combine the actual Q4 file you will receive with the cache/workspace after execution. The official weight of 12B works out to about 24GB in size, but this should not be mixed with the needs of a desktop Q4 file. The original and converted versions have different execution conditions.
What to check when memory is limited
If you have a 16GB device, you can test compatible 12B Q4 in a short context and in one request. On Mac, the operating system and other apps use the same memory, so success cannot be guaranteed just by looking at the display capacity.
26B-A4B requires more space to store full professional weights. In current devices, if some of this goes to RAM, the benefit of lower active computation can be overshadowed by transfer latency. Check the conditions and speed for sufficient memory.
When adding images and voices
After the text questions are answered well, add images or audio input one by one. Check the current model size and inputs your app supports, and see how memory and first token time vary. When input processing components are added, the leeway can be different than when you only see text.
You do not need to turn on all supported contexts from the beginning. Start with the actual document length and increase it to make sure you're finding the important content. The fact that you accepted a long document and the fact that you used it accurately until the end are two different things.
When there is a reason to change to a larger model
Send questions that are repeatedly incorrect in the small model to the large model. There's a reason to spend money on additional memory if it reduces actual errors and the wait is manageable. If the difference in results is small, you can choose to keep the smaller model with some margin.
There is no need to follow all the largest models in the new range. It's also a good idea to leave a stable model that handles your current documentation and code. Equipment comparison can be done after the model and input length have been determined.