Prefill and token generation speed
RTX 5090 32GB with Gemma 4 26B-A4B (MoE): prefill and generation speed
Last updated
1792 GB/s
Active 3.8B / Total 25.2B
122.7 ~ 284.8 tok/sDecode speed
Model fully loaded (15.55GB / 30.5GB)
First token (TTFT)
0.56 to 0.85 secondsContext 4,096 input tokens included
Prefill speed
4,856 ~ 7,408 tok/s0.55 to 0.84 seconds (Estimation based on public benchmarks)
Total time (E2E)
Approximately 0.96 to 1.8 seconds117 output tokens included
Press Play to process the input and generate the answer.
Ready
Expand settings Settings4-bit · 4K · Baseline decodeCollapse
QuantizationNeed 15.55GB
Input context4K tokens
Baseline
Baseline
Sample question:
117 output tokensCheck configuration
RTX 5090 32GB · Memory 32GB
Gemma 4 26B-A4B (MoE) Q4_K_M configuration. Check that the retailer lists the same memory option.
Already own it? Open the setup guideWould another device feel faster?
Compare the same model including prefill, or just token generation.
What changes with a longer document?Expand input length, waiting time, speed and memory
Detailed analysis and all 60 hardware configurations
Popular configurations
Speed simulations by device and model
Qwen3.8 27B on RTX 5090: recommended settings
148.2 ~ 217.8 tok/s · 0.27 to 0.39 seconds
Gemma 4 12B on Mac mini M4 24GB: chat speed
11.6 ~ 12.9 tok/s · 12.8 ~ 31.1 s
Qwen3.8 27B on Mac mini M4 32GB: response speed
5.1 ~ 5.7 tok/s · 16.4 ~ 35.5 s
Qwen3.8 27B: is a Mac mini M4 Pro 48GB worth it?
12.3 ~ 13.7 tok/s · 7.8 ~ 14.3 s